We all have used dataloader to import/export data in/out of Salesforce. There are scenarios where we need to automate it or scenarios where we need to upload different objects in a new Sandbox or Scratch org to make it ready for usage. We can also use SFDX but it can only upload 200 records per command.
I had a scenario where after every release we need to refresh a sandbox but also need to make it ready with some data so that testing can be done properly. We can also use Sandbox templates but that, only works with Full Copy or partial copy Sandbox. So the only choice left is Dataloader, but if you have 10+ object and couple of them master-detail and if you have a release every 2-4 weeks and developer sandbox then it becomes very time consuming task every time. So you need to find a way to auto populate all required data in one go. For which we can use Dataloader CLI
Steps to SetUp Dataloader CLI
Assuming you are using windows, either go to data loader installing location (salesforce\Data loder\bin) or add bin folder location to path variable. Now you need to generate an encrypted key for your operation.
.\encrypte.bat -e <text> <keyfile path location>
When data loader is installed a default key is generated in C:\Users\-username-\.dataloader folder with dataloader.key as name. Run the command from Dataloader\bin folder
.\encrypt.bat -e <sometext> C:\Users\--\.dataloader\dataloader.key
This will generate and encrypted key copy it and save it using notepate with name key.txt in your project folder. I will be using sample object name like Candidate, Position and Job Application in which Position and Job Application are master detail. My project folder looks like this…
First things first, encrypt your Salesfore password including security token with the generated key. Go to Dataloader Bin folder and open command prompt any type .\encrypt.bat -e yoursalesforcepassword C:\—\MyProject\key.txt . If you have proxy then encrypt the proxy password also. Copy and save this passwords because it will be used later
.sld files – are dataloader mapping files. Which maps the csv column name to salesforce field api name
#Mapping values for Canidate Name=Name Date of Birth=DataOfBirth__c Age=Age__c Email=Email__c
Key.txt – This file contains the encrypted key generated in first step
config.properties – contains configuration for dataloader. check below example
sfdc.debugMessages=false process.encryptionKeyFile=C:\\---MyProject\\key.txt sfdc.endpoint=https\://yourdomain.salesforce.com/ sfdc.username=username sfdc.password=<Encypted Salesforce Password> # Proxy info if any sfdc.proxyUsername=proxy username sfdc.proxyPassword=<proxy password encrypted> sfdc.proxyHost= yourproxy.com sfdc.proxyPort=0000 sfdc.loadBatchSize=200 sfdc.timeoutSecs=600
process-conf.xml – this is the file where you need to specify job for data loader it is a bean file where you have multiple beans to specify each job type
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd"> <beans> <!-- Candidate --> <bean id="candidateInsert" class="com.salesforce.dataloader.process.ProcessRunner" singleton="false"> <description>candidateInsert inserts caondidate records into Salesforce.</description> <property name="name" value="candidateInsert" /> <property name="configOverrideMap"> <map> <entry key="sfdc.entity" value="Candidate__c"/> <entry key="process.operation" value="insert"/> <entry key="process.mappingFile" value="C:\-\-MyProject\Candidate\candidate.sdl"/> <entry key="dataAccess.name" value="C:\-\-MyProject\Candidate\candidate.csv"/> <entry key="dataAccess.type" value="csvRead"/> <entry key="process.lastRunOutputDirectory" value="C:\-\-MyProject\Candidate\candidate\output"/> </map> </property> </bean> <!-- Position --> <bean id="positionInsert"> --- </bean> </beans>
process.bat – this is file where we instruct dataloader to execute multiple process configs
echo "Starting" set projectdir="C:\-\-\MyProject" cls cd "C:\Users\-\-\-\-\Data Loader\bin" CALL process.bat %projectdir% "candidateInsert" CALL process.bat %projectdir% "positionInsert" CALL process.bat %projectdir% "jobApplicationInsert" pause
You can Insert, Update, Upsert and Export data using dataloader cli. For more information check the salesforce dataloader guide or the samples which are available in the dataloader installation folder
Special Case (Master-Detail)
Master detail relations cannot be uploaded directly. For that you need to follow below steps.
- Create ExternlId field on parent object, map it with old salesforce id or just provide dummy unique values in csv file. Add the mapping in sdl file.
- In child object sdl mapping file. Taking Position as parent object and JobApplication as child example. In JobApplication sdl file for Position column use Position__r.ExternalId__c=Position__r\:ExternalId__c the column name should also be Position__r.ExternalId__c in csv
- Make sure Position process is run first in process.bat file
Finally, just double click the process.bat file and wait for the data to be uploaded