Definition:
Copying information from one or more sources to one or more targets.
Examples:
- a visitor registers on a Salesforce community, then a process calls a web service to validate his/her email address
- a Salesforce user clicks a button on an Account page, then a LWC calls Google Maps to calculate the directions from the user’s address to the account’s address
- a batch process in Salesforce collects closed won opportunities in the past 24 hours to send to a fulfillment service
- a third-party chat bot service calls a Salesforce web service to notify that a chat has been closed
- a process in another platform connects to Salesforce API to send new customer records
- an integration process in Mulesoft collects accounts created or updated since the last integration run and copies them to the marketing system
- a script to run DataLoader and upload CSV files from a folder to create records in Salesforce
Methods:
- API – callouts to services using HTTP protocol
- Middleware/ETL tools (Boomi, Informatica, Jitterbit, Mulesoft, …) that will deal with the APIs
Types of Integration:
UI integration
UI integration is not part of this document. The common scenario is of a user having to toggle between Salesforce and another application. An UI integration would make that toggle transparent. Example: CTI apps, hyperlinks, embedded iframes/canvas apps, web browser extensions, etc.
Point to point
One to one integration: your Salesforce org calls another service to pass or to fetch data (or another process calls a Salesforce web service)
It has a big disadvantage: each point will need a connection to the other point, more expensive to maintain since each pair is a separate integration
Hub and spoke
A central hub receives and sends data to the systems and is responsible for routing data between them.
Each system will only need one connection to the hub instead of a separate connection for each of the other systems.
The hub has to implement a generic business model, transform each application data to/from the generic model and deal with conflicts, ambiguities and compatibility issues.
Enterprise Service Bus
Evolution of a hub and spoke: can do routing between one or more systems and orchestrate the order in which the integrations occur.
Types of Data Synchronization:
- Unidirectional – data flows from source/master to target
- Bidirectional – data flows from one direction then the reverse direction with special handling to not overwrite each other’s updates or adopting a policy of “the latest update wins”
- Two-way-unidirectional – object is synced in both directions but in different fields and there is no overlap of data
Integration patterns in Salesforce:
- request and reply – synchronous, the remote service responds quickly
- fire and forget – asynchronous, the remote service takes long to respond and we can’t wait
- batch data sync – asynchronous, involves larger amounts of data to be copied during off hours
- remote call-in – remote system calls Salesfoce APIs or Apex services to enter data
- UI update – event-driven, user is notified on-screen when relevant data changes
- data virtualization – external objects using Salesforce Connect
Questions to ask when designing an integration
Think of it as both the future ongoing integration and the initial data load as well:
- How and when the data will be ready? What is the format of the data?
- Who is the subject matter expert?
- Are there any security concerns or guidelines?
- What middleware/ETL tool to use?
- How often should the data be copied and in what directions (unidirectional, bidirectional, two-way-unidirectional, to Salesforce, from Salesforce)?
- Request integration user credentials to Salesforce (dev/UAT/Prod) and their other systems.
- What Salesforce objects are to be mapped to what external tables/services?
- When the mapping document will be ready for review and who is responsible for it?
- When will the target SF org be ready? (UAT or Prod)
- Who will do the UAT/QA and what is the org?
- How long should it take to go live?
Tools/items that help with integration:
- jwt.io
- Postman, Insomnia
- SOAP.ui
- Salesforce Workbench
Salesforce implementation items that help with integration:
- Ability to disable triggers/flows using custom setting for the integration user
- Store external system Ids as external ids in Salesforce (or vice-versa: store Salesforce Ids in the external systems)
- If a record is inserted/updated in Salesforce during integration, have triggers take care of wiring parent/children records to it instead of the process in the middleware
- Sort children records by parent to avoid record locking errors
- Have a “Source” field to store the name of the system where the record was originated
- Have a “Job Id” field to store the integration’s job id: this identifies records updated in the last run of the integration
- Have the integration process use upsert (or check before inserting/updating) to avoid duplicates
- Log integration errors in a Salesforce object named Error_Log__c or Integration_Error__c
Other things in Salesforce that pertain to integration:
- Named Credentials
- Remote Site Settings
- Workflow Outbound Messages
- Salesforce/Lightning Connect, Heroku Connect, Apex Callouts
- SOAP API, REST API, Bulk API, Streaming API
Sources/Further study:
- https://www.researchgate.net/publication/47394009_An_integration_strategy_for_large_enterprises
- https://www.centeractive.com/content/enterprise-service-bus
- https://www.salesforceben.com/salesforce-integration/
- https://developer.salesforce.com/docs/atlas.en-us.integration_patterns_and_practices.meta/integration_patterns_and_practices/integ_pat_remote_process_invocation_state.htm
- https://resources.docs.salesforce.com/sfdc/pdf/integration_patterns_and_practices.pdf
- https://trailhead.salesforce.com/en/content/learn/superbadges/superbadge_integration
- https://resources.docs.salesforce.com/sfdc/pdf/salesforce_large_data_volumes_bp.pdf
- Developing Data Migrations and Integrations with Salesforce – David Masri
Personal experiences:
- Most common objects to integrate are (in decreasing order): Account, Contact, Opportunity, then Product, Pricebook, PricebookEntry, OpportunityLineItem, Quote, QuoteLineItem
- Most common issue in integrations: client doesn’t have an expert on their source/target system (or lacks availability)
- Second most common issues: bad API (doesn’t have pagination, filtering, batching), cumbersome API (requires a lot of separate calls or returns cryptic error messages or stack dump instead of error message), lack of documentation
- Demand for integration consulting is ever increasing, up to 3x times from one year to the next