{"id":601,"date":"2019-02-15T12:53:00","date_gmt":"2019-02-15T18:53:00","guid":{"rendered":"https:\/\/blog.felineflock.com\/?p=601"},"modified":"2024-01-01T13:01:19","modified_gmt":"2024-01-01T19:01:19","slug":"how-to-quickly-copy-production-data-for-testing","status":"publish","type":"post","link":"https:\/\/blog.felineflock.com\/index.php\/2019\/02\/15\/how-to-quickly-copy-production-data-for-testing\/","title":{"rendered":"How to quickly copy production data for testing"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Salesforce CLI has command options to export\/import data hierarchically. For example, you can export a few accounts plus their respective opportunities in a tree structure and import them that way without having to manually rewire them with the new IDs in the target org.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here are 4 SFDX commands you can run on Terminal\/Command prompt to do just that:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">sfdx force:auth:web:login -d -a\u00a0myProductionOrg <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">sfdx force:data:tree:export -q &#8220;SELECT ID, Name, \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     ( SELECT ID, Name, StageName, AccountID, \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     CloseDate, Amount FROM Opportunities ) \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     FROM Account WHERE Name IN ( \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     &#8216;Nice Account with oppties to test with&#8217;, \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     &#8216;Another good account&#8217; ) &#8221; \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     -u myProductionOrg &#8211;outputdir .\/TestDataFromProd &#8211;plan <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">sfdx force:auth:web:login -d -a\u00a0mySandbox\u00a0-r https:\/\/test.salesforce.com <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">sfdx force:data:tree:import -u\u00a0mySandbox\u00a0 \\<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">     &#8211;plan .\/TestDataFromProd\/Account-Opportunity-plan.json<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">The first command (<strong>sfdx force:auth:web:login<\/strong>) opens the browser for you to log into Production and authorize SFDX.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The second command (<strong>sfdx force:data:tree:export<\/strong>) exports the query result into files in the TestDataFromProd folder. It will create these 3 files:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">Account-Opportunity-plan.json <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Accounts.json <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Opportunitys.json<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">SFDX will export up to 2000 records in one file per object plus a plan file to link accounts and opportunities. When importing, there is a limit of 200 records per file so you may have to split files (more on that later).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The third command (<strong>sfdx force:auth:web:login<\/strong>) opens the browser for you to log into the sandbox and authorize SFDX there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The last command (<strong>sfdx force:data:tree:import<\/strong>) imports the files from the TestDataFromProd folder into the sandbox and wires up accounts with respective opportunities at the same time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In case you need to split the JSON file, I&#8217;ve adapted the Node script below:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">if( process.argv.length &lt; 3 ) {<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 console.log( &#8216;Target JSON file is required.&#8217; );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 process.exit( 1 );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">} <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">var target = process.argv[ 2 ]; <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">console.log( &#8216;File: &#8216; + target ); <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">var fs = require( &#8216;fs&#8217; ); <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">var newFileName = target.replace( &#8220;.json&#8221;, &#8220;&#8221; ); <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">fs.readFile( target, function ( err, data ) {<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 if (err) throw err;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 var jsonArray = JSON.parse( data );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 var i = 1;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 jsonArray = jsonArray.records;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 while( jsonArray.length !== 0 ) {<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 var fileName = newFileName + &#8216;-&#8216; + i + &#8220;.json&#8221;;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 var newArray = jsonArray.splice( 0, 200 );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 var newObj = { records: newArray };<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 fs.writeFileSync( fileName, JSON.stringify( newObj ) );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 console.log( fileName );<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0  \u00a0 \u00a0 i++;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 }<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> })<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">You can run it like below in OS X:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">node .\/split-json.js Opportunitys.json<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">It will split Opportunitys.json into numbered files: Opportunitys-1.json, Opportunitys-2.json, Opportunitys-3.json, &#8230; each having up to 200 records.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Before importing, edit the plan file (Account-Opportunity-plan.json) to add the names of the new split files in place of the original Opportunitys.json file:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\">{<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 &#8220;sobject&#8221;: &#8220;Opportunity&#8221;,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 &#8220;saveRefs&#8221;: false,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 &#8220;resolveRefs&#8221;: true,<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 &#8220;files&#8221;: [<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 &#8220;Opportunitys-1.json&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 , &#8220;Opportunitys-2.json&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"> \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 , &#8220;Opportunitys-3.json&#8221; &#8230;<\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">The structure of SObject trees accommodates 5 levels deep so we should be able to import accounts with opportunities and line items:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/developer.salesforce.com\/docs\/atlas.en-us.api_rest.meta\/api_rest\/resources_composite_sobject_tree.htm\">https:\/\/developer.salesforce.com\/docs\/atlas.en-us.api_rest.meta\/api_rest\/resources_composite_sobject_tree.htm<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, queries can only bring parent and child relationships &#8211; only one level of nested records, that is, accounts and their opportunities, but not the line items in each opportunity &#8211; so that still needs to be researched.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It is possible to create plugins for SFDX so that looks like one way to implement the ability to get deeper SObject trees to import:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/forcedotcom\/sfdx-plugin-generate\">https:\/\/github.com\/forcedotcom\/sfdx-plugin-generate<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another way it may be possible is to run a second query export (say, opportunities and their line items) and edit the first plan file to add more steps. There may be a challenge in writing both queries to bring opportunities in the same order.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Salesforce CLI has command options to export\/import data hierarchically. For example, you can export a few accounts plus their respective opportunities in a tree structure and import them that way without having to manually rewire them with the new IDs in the target org. Here are 4 SFDX commands you can run on Terminal\/Command prompt [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-601","post","type-post","status-publish","format-standard","hentry","category-blog"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/601","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/comments?post=601"}],"version-history":[{"count":3,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/601\/revisions"}],"predecessor-version":[{"id":604,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/posts\/601\/revisions\/604"}],"wp:attachment":[{"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/media?parent=601"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/categories?post=601"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.felineflock.com\/index.php\/wp-json\/wp\/v2\/tags?post=601"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}