{"id":637,"date":"2022-02-07T20:37:00","date_gmt":"2022-02-08T02:37:00","guid":{"rendered":"https:\/\/blog.felineflock.com\/?p=637"},"modified":"2024-01-03T20:43:28","modified_gmt":"2024-01-04T02:43:28","slug":"data-integration-in-salesforce-a-quick-and-simple-overview","status":"publish","type":"post","link":"https:\/\/blog.felineflock.com\/index.php\/2022\/02\/07\/data-integration-in-salesforce-a-quick-and-simple-overview\/","title":{"rendered":"Data Integration in Salesforce: A Quick and Simple Overview"},"content":{"rendered":"\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"737\" height=\"113\" src=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-14.png\" alt=\"\" class=\"wp-image-638\" srcset=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-14.png 737w, https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-14-300x46.png 300w\" sizes=\"auto, (max-width: 737px) 100vw, 737px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1167\"><strong>Definition<\/strong>:<\/h2>\n\n\n\n<p id=\"ember1168\">Copying information from one or more sources to one or more targets.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1170\">Examples:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>a visitor registers on a Salesforce community, then a process calls a web service to validate his\/her email address<\/li>\n\n\n\n<li>a Salesforce user clicks a button on an Account page, then a LWC calls Google Maps to calculate the directions from the user&#8217;s address to the account&#8217;s address<\/li>\n\n\n\n<li>a batch process in Salesforce collects closed won opportunities in the past 24 hours to send to a fulfillment service<\/li>\n\n\n\n<li>a third-party chat bot service calls a Salesforce web service to notify that a chat has been closed<\/li>\n\n\n\n<li>a process in another platform connects to Salesforce API to send new customer records<\/li>\n\n\n\n<li>an integration process in Mulesoft collects accounts created or updated since the last integration run and copies them to the marketing system<\/li>\n\n\n\n<li>a script to run DataLoader and upload CSV files from a folder to create records in Salesforce<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1173\"><strong>Methods<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>API &#8211; callouts to services using HTTP protocol<\/li>\n\n\n\n<li>Middleware\/ETL tools (Boomi, Informatica, Jitterbit, Mulesoft, \u2026) that will deal with the APIs<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1176\"><strong>Types of Integration<\/strong>:<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"ember1177\">UI integration<\/h3>\n\n\n\n<p id=\"ember1178\"><strong>UI integration is not part of this document. <\/strong>The common scenario is of a user having to toggle between Salesforce and another application. An UI integration would make that toggle transparent. Example:&nbsp; CTI apps, hyperlinks, embedded iframes\/canvas apps, web browser extensions, etc.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"ember1180\">Point to point<\/h3>\n\n\n\n<p id=\"ember1181\">One to one integration: your Salesforce org calls another service to pass or to fetch data (or another process calls a Salesforce web service)<\/p>\n\n\n\n<p id=\"ember1182\">It has a big disadvantage:&nbsp; each point will need a connection to the other point, more expensive to maintain since each pair is a separate integration<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"474\" height=\"335\" src=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-16.png\" alt=\"\" class=\"wp-image-640\" srcset=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-16.png 474w, https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-16-300x212.png 300w\" sizes=\"auto, (max-width: 474px) 100vw, 474px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"ember1185\">Hub and spoke<\/h3>\n\n\n\n<p id=\"ember1186\">A central hub receives and sends data to the systems and is responsible for routing data between them.<\/p>\n\n\n\n<p id=\"ember1187\">Each system will only need one connection to the hub instead of a separate connection for each of the other systems.<\/p>\n\n\n\n<p id=\"ember1188\">The hub has to implement a generic business model, transform each application data to\/from the generic model and deal with conflicts, ambiguities and compatibility issues.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"320\" height=\"320\" src=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-17.png\" alt=\"\" class=\"wp-image-641\" srcset=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-17.png 320w, https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-17-300x300.png 300w, https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-17-150x150.png 150w\" sizes=\"auto, (max-width: 320px) 100vw, 320px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"ember1191\">Enterprise Service Bus<\/h3>\n\n\n\n<p id=\"ember1192\">Evolution of a hub and spoke:&nbsp; can do routing between one or more systems and orchestrate the order in which the integrations occur.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"520\" height=\"388\" src=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-18.png\" alt=\"\" class=\"wp-image-642\" srcset=\"https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-18.png 520w, https:\/\/blog.felineflock.com\/wp-content\/uploads\/2024\/01\/image-18-300x224.png 300w\" sizes=\"auto, (max-width: 520px) 100vw, 520px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1195\"><strong>Types of Data Synchronization<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Unidirectional &#8211; data flows from source\/master to target<\/li>\n\n\n\n<li>Bidirectional &#8211; data flows from one direction then the reverse direction with special handling to not overwrite each other&#8217;s updates or adopting a policy of &#8220;the latest update wins&#8221;<\/li>\n\n\n\n<li>Two-way-unidirectional &#8211; object is synced in both directions but in different fields and there is no overlap of data<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1198\"><strong>Integration patterns in Salesforce<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>request and reply &#8211; synchronous, the remote service responds quickly<\/li>\n\n\n\n<li>fire and forget &#8211; asynchronous, the remote service takes long to respond and we can&#8217;t wait<\/li>\n\n\n\n<li>batch data sync &#8211; asynchronous, involves larger amounts of data to be copied during off hours<\/li>\n\n\n\n<li>remote call-in &#8211; remote system calls Salesfoce APIs or Apex services to enter data<\/li>\n\n\n\n<li>UI update &#8211; event-driven, user is notified on-screen when relevant data changes<\/li>\n\n\n\n<li>data virtualization &#8211; external objects using Salesforce Connect<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1201\"><strong>Questions to ask when designing an integration<\/strong><\/h2>\n\n\n\n<p id=\"ember1202\">Think of it as both the future ongoing integration and the initial data load as well:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How and when the data will be ready? What is the format of the data?<\/li>\n\n\n\n<li>Who is the subject matter expert?<\/li>\n\n\n\n<li>Are there any security concerns or guidelines?<\/li>\n\n\n\n<li>What middleware\/ETL tool to use?<\/li>\n\n\n\n<li>How often should the data be copied and in what directions (unidirectional, bidirectional, two-way-unidirectional, to Salesforce, from Salesforce)?<\/li>\n\n\n\n<li>Request integration user credentials to Salesforce (dev\/UAT\/Prod) and their other systems.<\/li>\n\n\n\n<li>What Salesforce objects are to be mapped to what external tables\/services?<\/li>\n\n\n\n<li>When the mapping document will be ready for review and who is responsible for it?<\/li>\n\n\n\n<li>When will the target SF org be ready? (UAT or Prod)<\/li>\n\n\n\n<li>Who will do the UAT\/QA and what is the org?<\/li>\n\n\n\n<li>How long should it take to go live?<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1205\"><strong>Tools\/items that help with integration<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>jwt.io<\/li>\n\n\n\n<li>Postman, Insomnia<\/li>\n\n\n\n<li>SOAP.ui<\/li>\n\n\n\n<li>Salesforce Workbench<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1208\"><strong>Salesforce implementation items that help with integration<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Ability to disable triggers\/flows using custom setting for the integration user<\/li>\n\n\n\n<li>Store external system Ids as external ids in Salesforce (or vice-versa: store Salesforce Ids in the external systems)<\/li>\n\n\n\n<li>If a record is inserted\/updated in Salesforce during integration, have triggers take care of wiring parent\/children records to it instead of the process in the middleware<\/li>\n\n\n\n<li>Sort children records by parent to avoid record locking errors<\/li>\n\n\n\n<li>Have a &#8220;Source&#8221; field to store the name of the system where the record was originated<\/li>\n\n\n\n<li>Have a &#8220;Job Id&#8221; field to store the integration&#8217;s job id:&nbsp; this identifies records updated in the last run of the integration<\/li>\n\n\n\n<li>Have the integration process use upsert (or check before inserting\/updating) to avoid duplicates<\/li>\n\n\n\n<li>Log integration errors in a Salesforce object named Error_Log__c or Integration_Error__c<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1211\"><strong>Other things in Salesforce that pertain to integration<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Named Credentials<\/li>\n\n\n\n<li>Remote Site Settings<\/li>\n\n\n\n<li>Workflow Outbound Messages<\/li>\n\n\n\n<li>Salesforce\/Lightning Connect, Heroku Connect, Apex Callouts<\/li>\n\n\n\n<li>SOAP API, REST API, Bulk API, Streaming API<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1214\"><strong>Sources\/Further study<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.researchgate.net\/publication\/47394009_An_integration_strategy_for_large_enterprises\">https:\/\/www.researchgate.net\/publication\/47394009_An_integration_strategy_for_large_enterprises<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.centeractive.com\/content\/enterprise-service-bus\">https:\/\/www.centeractive.com\/content\/enterprise-service-bus<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.salesforceben.com\/salesforce-integration\/\">https:\/\/www.salesforceben.com\/salesforce-integration\/<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/developer.salesforce.com\/docs\/atlas.en-us.integration_patterns_and_practices.meta\/integration_patterns_and_practices\/integ_pat_remote_process_invocation_state.htm\">https:\/\/developer.salesforce.com\/docs\/atlas.en-us.integration_patterns_and_practices.meta\/integration_patterns_and_practices\/integ_pat_remote_process_invocation_state.htm<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/resources.docs.salesforce.com\/sfdc\/pdf\/integration_patterns_and_practices.pdf\">https:\/\/resources.docs.salesforce.com\/sfdc\/pdf\/integration_patterns_and_practices.pdf<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/trailhead.salesforce.com\/en\/content\/learn\/superbadges\/superbadge_integration\">https:\/\/trailhead.salesforce.com\/en\/content\/learn\/superbadges\/superbadge_integration<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/resources.docs.salesforce.com\/sfdc\/pdf\/salesforce_large_data_volumes_bp.pdf\">https:\/\/resources.docs.salesforce.com\/sfdc\/pdf\/salesforce_large_data_volumes_bp.pdf<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.amazon.com\/Developing-Data-Migrations-Integrations-Salesforce\/dp\/1484242084\/\">Developing Data Migrations and Integrations with Salesforce &#8211; David Masri<\/a><\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"ember1217\"><strong>Personal experiences<\/strong>:<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Most common objects to integrate are (in decreasing order):&nbsp; Account, Contact, Opportunity, then Product, Pricebook, PricebookEntry, OpportunityLineItem, Quote, QuoteLineItem<\/li>\n\n\n\n<li>Most common issue in integrations:&nbsp; client doesn&#8217;t have an expert on their source\/target system (or lacks availability)<\/li>\n\n\n\n<li>Second most common issues:&nbsp; bad API (doesn&#8217;t have pagination, filtering, batching), cumbersome API (requires a lot of separate calls or returns cryptic error messages or stack dump instead of error message), lack of documentation<\/li>\n\n\n\n<li>Demand for integration consulting is ever increasing, up to 3x times from one year to the next<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Definition: Copying information from one or more sources to one or more targets. Examples: Methods: Types of Integration: UI integration UI integration is not part of this document. The common scenario is of a user having to toggle between Salesforce and another application. An UI integration would make that toggle transparent. Example:&nbsp; CTI apps, hyperlinks, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-637","post","type-post","status-publish","format-standard","hentry","category-blog"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/637","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/comments?post=637"}],"version-history":[{"count":1,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/637\/revisions"}],"predecessor-version":[{"id":643,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/637\/revisions\/643"}],"wp:attachment":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/media?parent=637"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/categories?post=637"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/tags?post=637"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}