In this post, I am going to show how to use Salesforce Bulk API which provides a programmatic option to quickly load your org’s data into Salesforce.
With the Bulk API, You process a set of records by creating a job that contains one or more batches. The job specifies which object is being processed and what type of operation is being used. A batch is a set of records sent to the server to process and Each batch is processed independently by the server, not necessarily in the order it is received. Batches may be processed in parallel or concurrent. Here are the details steps to load the data with bulk API
Step 1: Log In Using the SOAP API
The Bulk API doesn’t provide a login operation, so you must use SOAP API to log in.Create a text file called login.txt containing the following text and replace the username and password.
<?xml version="1.0" encoding="utf-8" ?>
<env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Body>
<n1:login xmlns:n1="urn:partner.soap.sforce.com">
<n1:username>your_username</n1:username>
<n1:password>your_password</n1:password>
</n1:login>
</env:Body>
</env:Envelope>
Run the following command from the cURL command line.
curl https://login.salesforce.com/services/Soap/u/42.0 -H "Content-Type: text/xml; charset=UTF-8" -H "SOAPAction: login" -d @login.txt
Which will return the an XML response that includes <sessionId> and elements. We are going to use the sessionId for the subsequent request authentication.
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns="urn:partner.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<loginResponse>
<result>
<metadataServerUrl>https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/Soap/m/42.0/00D6A000002EfbH</metadataServerUrl>
<passwordExpired>false</passwordExpired>
<sandbox>false</sandbox>
<serverUrl>https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/Soap/u/42.0/00D6A000002EfbH</serverUrl>
<sessionId>00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1</sessionId>
<userId>0056A000001dwToQAI</userId>
<userInfo>
<accessibilityMode>false</accessibilityMode>
<chatterExternal>false</chatterExternal>
<currencySymbol>$</currencySymbol>
<orgAttachmentFileSizeLimit>5242880</orgAttachmentFileSizeLimit>
<orgDefaultCurrencyIsoCode>USD</orgDefaultCurrencyIsoCode>
<orgDefaultCurrencyLocale>en_US</orgDefaultCurrencyLocale>
<orgDisallowHtmlAttachments>false</orgDisallowHtmlAttachments>
<orgHasPersonAccounts>false</orgHasPersonAccounts>
<organizationId>00D6A000002EfbHUAS</organizationId>
<organizationMultiCurrency>false</organizationMultiCurrency>
<organizationName>Alumni Center Services</organizationName>
<profileId>00e6A000001YVOQQA4</profileId>
<roleId xsi:nil="true" />
<sessionSecondsValid>7200</sessionSecondsValid>
<userDefaultCurrencyIsoCode xsi:nil="true" />
<userEmail>rajamohanvakati@gmail.com</userEmail>
<userFullName>Rajamohan vakati</userFullName>
<userId>0056A000001dwToQAI</userId>
<userLanguage>en_US</userLanguage>
<userLocale>en_US</userLocale>
<userName>rvakati@spring18.com</userName>
<userTimeZone>America/Los_Angeles</userTimeZone>
<userType>Standard</userType>
<userUiSkin>Theme3</userUiSkin>
</userInfo>
</result>
</loginResponse>
</soapenv:Body>
</soapenv:Envelope>
Step 2: Create a Job
Before you can load the data, you need to create a job. The job specifies the type of object, such as Contact, that you’re loading and the operation that you’re performing, such as query, queryAll, insert, update, upsert, or delete. Create a text file called job.txt containing the following text.
<?xml version="1.0" encoding="UTF-8"?> <jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <operation>insert</operation> <object>Contact</object> <contentType>CSV</contentType> </jobInfo>
Now Run the following command from the cURL command line
curl https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/async/42.0/job -H "X-SFDC-Session: 00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1" -H "Content-Type: application/xml; charset=UTF-8" -d @job.txt
You will get the response as shown below. In this response, you will get the jobId which we will use in subsequence calls.
<?xml version="1.0" encoding="UTF-8"?><jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <id>7506A0000048yPlQAI</id> <operation>insert</operation> <object>Contact</object> <createdById>0056A000001dwToQAI</createdById> <createdDate>2018-03-26T22:03:19.000Z</createdDate> <systemModstamp>2018-03-26T22:03:19.000Z</systemModstamp> <state>Open</state> <concurrencyMode>Parallel</concurrencyMode> <contentType>CSV</contentType> <numberBatchesQueued>0</numberBatchesQueued> <numberBatchesInProgress>0</numberBatchesInProgress> <numberBatchesCompleted>0</numberBatchesCompleted> <numberBatchesFailed>0</numberBatchesFailed> <numberBatchesTotal>0</numberBatchesTotal> <numberRecordsProcessed>0</numberRecordsProcessed> <numberRetries>0</numberRetries> <apiVersion>42.0</apiVersion> <numberRecordsFailed>0</numberRecordsFailed> <totalProcessingTime>0</totalProcessingTime> <apiActiveProcessingTime>0</apiActiveProcessingTime> <apexProcessingTime>0</apexProcessingTime> </jobInfo>
Step 3: Add a Batch to the Job
After creating the job, you need to create the batch that contains the data to be loaded into the salesforce. Create a CSV file that contains the data. Using a command-line window, execute the following cURL command.
curl https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/async/42.0/job/7506A0000048yPlQAI/batch -H "X-SFDC-Session: 00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1" -H "Content-Type: text/csv; charset=UTF-8" --data-binary @data.csv
Which will return the batchId as shown below.
<?xml version="1.0" encoding="UTF-8"?><batchInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <id>7516A000003R4blQAC</id> <jobId>7506A0000048yPlQAI</jobId> <state>Queued</state> <createdDate>2018-03-26T22:17:53.000Z</createdDate> <systemModstamp>2018-03-26T22:17:53.000Z</systemModstamp> <numberRecordsProcessed>0</numberRecordsProcessed> <numberRecordsFailed>0</numberRecordsFailed> <totalProcessingTime>0</totalProcessingTime> <apiActiveProcessingTime>0</apiActiveProcessingTime> <apexProcessingTime>0</apexProcessingTime> </batchInfo>
Step 4: Close the Job
When you’re finished submitting batches to Salesforce, close the job. This informs Salesforce that no more batches will be submitted for the job, which, in turn, allows the monitoring page in Salesforce to return more meaningful statistics on the progress of the job.Create a text file called close_job.txt containing the following text.
<?xml version="1.0" encoding="UTF-8"?> <jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <state>Closed</state> </jobInfo>
Run the following command from cURL
curl https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/async/42.0/job/7506A0000048yPlQAI -H "X-SFDC-Session: 00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1" -H "Content-Type: application/xml; charset=UTF-8" -d @close_job.txt
Which will return the batch details and update the job resource state from Open to Closed.
<?xml version="1.0" encoding="UTF-8"?><jobInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <id>7506A0000048yPlQAI</id> <operation>insert</operation> <object>Contact</object> <createdById>0056A000001dwToQAI</createdById> <createdDate>2018-03-26T22:03:19.000Z</createdDate> <systemModstamp>2018-03-26T22:03:19.000Z</systemModstamp> <state>Closed</state> <concurrencyMode>Parallel</concurrencyMode> <contentType>CSV</contentType> <numberBatchesQueued>0</numberBatchesQueued> <numberBatchesInProgress>0</numberBatchesInProgress> <numberBatchesCompleted>1</numberBatchesCompleted> <numberBatchesFailed>0</numberBatchesFailed> <numberBatchesTotal>1</numberBatchesTotal> <numberRecordsProcessed>26</numberRecordsProcessed> <numberRetries>0</numberRetries> <apiVersion>42.0</apiVersion> <numberRecordsFailed>2</numberRecordsFailed> <totalProcessingTime>186</totalProcessingTime> <apiActiveProcessingTime>105</apiActiveProcessingTime> <apexProcessingTime>0</apexProcessingTime> </jobInfo>
Step 5: Check Batch Status
You can check the status of an individual batch by running the following cURL command.
curl https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/async/42.0/job/7506A0000048yPlQAI/batch/7516A000003R4blQAC -H "X-SFDC-Session: 00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1"
which will return the following XML response with the status like Failed or completed.
<?xml version="1.0" encoding="UTF-8"?><batchInfo xmlns="http://www.force.com/2009/06/asyncapi/dataload"> <id>7516A000003R4blQAC</id> <jobId>7506A0000048yPlQAI</jobId> <state>Completed</state> <createdDate>2018-03-26T22:17:53.000Z</createdDate> <systemModstamp>2018-03-26T22:17:53.000Z</systemModstamp> <numberRecordsProcessed>26</numberRecordsProcessed> <numberRecordsFailed>2</numberRecordsFailed> <totalProcessingTime>186</totalProcessingTime> <apiActiveProcessingTime>105</apiActiveProcessingTime> <apexProcessingTime>0</apexProcessingTime> </batchInfo>
Step 6: Retrieve Batch Results
Once a batch is Completed, you need to retrieve the batch result to see the status of individual records. Retrieve the results for an individual batch by running the following cURL command:
curl https://asdasdasdasdasd-dev-ed.my.salesforce.com/services/async/42.0/job/7506A0000048yPlQAI/batch/7516A000003R4blQAC/result -H "X-SFDC-Session: 00D6A000002EfbH!AQwAQIEc6HU3xtluZmjmPEdRzpCZVk.PsdS.Lbpp.NT5W7QL8NHEFcTtIyvKhdum7.6zHxgAVFzRxFdB481_cLYSQiyFrzb1"
Salesforce returns a response with data such as the following
"Id","Success","Created","Error" "0036A00000QniplQAB","true","true","" "0036A00000QnipmQAB","true","true","" "0036A00000QnipnQAB","true","true","" "0036A00000QnipoQAB","true","true","" "0036A00000QnippQAB","true","true","" "0036A00000QnipqQAB","true","true","" "0036A00000QniprQAB","true","true","" "0036A00000QnipsQAB","true","true","" "0036A00000QniptQAB","true","true","" "","false","false","INVALID_EMAIL_ADDRESS:Email: invalid email address: b.levy@expressl&t.net:Email --" "","false","false","INVALID_EMAIL_ADDRESS:Email: invalid email address: j.davis@expressl&t.net:Email --" "0036A00000QnipuQAB","true","true","" "0036A00000QnipvQAB","true","true","" "0036A00000QnipwQAB","true","true","" "0036A00000QnipxQAB","true","true","" "0036A00000QnipyQAB","true","true","" "0036A00000QnipzQAB","true","true","" "0036A00000Qniq0QAB","true","true","" "0036A00000Qniq1QAB","true","true","" "0036A00000Qniq2QAB","true","true","" "0036A00000Qniq3QAB","true","true","" "0036A00000Qniq4QAB","true","true","" "0036A00000Qniq5QAB","true","true","" "0036A00000Qniq6QAB","true","true","" "0036A00000Qniq7QAB","true","true","" "0036A00000Qniq8QAB","true","true",""
Beyond Cloud 