Salesforce Bulk API 2.0

On By Rajamohan VakatiIn Bulk API

Introduction

In this blog, I am going to show how to use Salesforce Bulk API 2.0.Bulk API 2.0 is generally available in Winter ’18.Bulk API v2 brings the power of bulk transactions from Bulk API v1 into a simplified, easier-to-use REST API. Bulk API v2 lets you create, update, or delete millions of records asynchronously by using REST API.
Here are the features of the Salesforce Bulk API 2.0

  • Bulk API 2.0 supports all OAuth flows supported by Salesforce’s other REST APIs.
  • Bulk API 2.0 simplifies uploading large amounts of data by breaking the data into batches automatically. All you have to do is upload a CSV file with your record data and check back when the results are ready.
  • Instead of limiting the amount of data uploaded daily by the number of jobs, Bulk API 2.0 uses a limit of total records uploaded. The limit is 100 million records per 24-hour period.
  • Bulk API v2 does away with the need to manually break up data into batches.

Data Processing 

Salesforce processes Bulk API 2.0 jobs you create and uses job state to describe the status of a queued job.Salesforce queues a new job for processing once you’ve created it, uploaded data for the job, and moved the job to the UploadComplete state. Once the job is de-queued, Salesforce starts processing the job operation and moves the job to the InProgress state. Once the job has been fully processed, Salesforce moves the job to the JobComplete state, or Failed state,
depending on if the job was successfully processed or not.Salesforce automatically chunks the data for your job into multiple internal batches to improve performance while processing the job.Salesforce creates a separate batch for every 10,000 records in your job data, up to a daily maximum of 100 million records. If the limit is exceeded while processing your job data, the remaining data isn’t processed.
For each internal batch, if Salesforce can’t process all the records in a batch in 10 minutes or less, the batch fails. Salesforce will re-try failed batches up to a maximum of 10 retries. If the batch still can’t be processed after 10 retries, the entire job is moved to the Failed state and remaining job data is not processed. Use the Failed Record Results and Unprocessed Record Records resources to determine
what records were not processed and what errors occurred.

Bulk API v2.0 Example 

Bulk API 2.0 data processing contains the below steps

  1. Authenticate with REST API. We are using the workbench here
  2. Create a job.
  3. Upload data for the job.
  4. close the job by setting the job state to UploadComplete. Salesforce will start processing the job at this point.
  5. Check the status of the job.
  6. Retrieve the results of the completed job by checking the successful results, failed results, and unprocessed records.

1.Login into workbench
go to https://workbench.developerforce.com/ link and login into the workbench with Salesforce
2.Create a job
To do any Bulk API 2.0 task, such as inserting or updating records, you first create a job. The job specifies the type of object, such as Account, that you’re loading. The job also specifies the operation you’re performing, such as insert or delete.In workbench  Go to Utilities > REST Explorer and create a job by issuing a POST request with the following details as shown below

Once you execute the REST Call, You should get a response that includes the job ID, with a job state of Open. You’ll use the job ID in Bulk API 2.0 calls in the next
steps. You’ll also use the URL in the contentUrl field of the response in the next step when you upload your data.

3.Upload data for the job

After creating a job, you’re ready to upload your data. You provide record data using the CSV file you created earlier.In workbench’s REST Explorer, use Bulk API 2.0 to create a job data request with the CSV data. Issue a PUT request with the following
details. For the Request Body, copy and paste the CSV data into workbench’s Request Body text field. With Bulk API 2.0, you can submit CSV
data that does not in total exceed 150 MB in size (after base64 encoding).

4. Close the job.

Once you’re done submitting data, you can inform Salesforce that the job is ready for processing by closing the job. In workbench’s REST Explorer, issue a PATCH request with the following details.

5. Check the job status and results.

To get basic status information on a job, such as the overall job state or the number of records processed, use a GET request with the following details:

6.Retrieve the results 

Once a job has been completed and is in the JobComplete state (or Failed state), you can get details of which job data records were successfully processed by issuing a GET request with the following details:

/services/data/vXX.X/jobs/ingest/JOB ID/successfulResults/

You get a response that contains CSV data, with each row containing a record ID and information on whether that record was successfully processed or not. To get details on records that encountered an error during processing, use a GET request using the failedResults resource instead of the successfulResults resource.