{"id":110804,"date":"2017-04-28T06:27:39","date_gmt":"2017-04-28T13:27:39","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/power-platform\/blog\/power-automate\/copying-using-filtering\/"},"modified":"2017-04-28T06:27:39","modified_gmt":"2017-04-28T13:27:39","slug":"copying-using-filtering","status":"publish","type":"power-automate","link":"https:\/\/www.microsoft.com\/en-us\/power-platform\/blog\/power-automate\/copying-using-filtering\/","title":{"rendered":"Copy data between two SharePoint lists (or any two data sources) efficiently"},"content":{"rendered":"
One common scenario in Microsoft Flow is copying data between two different data sources. For example, if you have a SharePoint list in one site, and you want all the items to be copied over to a second SharePoint site, you can build a flow to do that. This post will walk you through how to do this efficiently as possible.<\/p>\n
In this scenario there are two lists SharePoint lists (but the same process would work between a SharePoint list and a SQL table or any number of the 100+ services supported by Microsoft Flow). It’s also not required that the names of all of the columns are the same, but you will need to be able to populate all of the required columns.<\/p>\n
Finally, there has to be at least one column that will be able to uniquely identify items between the two SharePoint lists. In the example below, we will assume that the Title<\/strong> column is the same, but it could be any column you want.<\/p>\n The first step is to set up your trigger on the source SharePoint list. Because you want to get all item changes (not just new items) you should select the\u00a0Sharepoint –\u00a0When an existing item is modified<\/strong> trigger.\u00a0<\/p>\n <\/p>\n Note that despite the name, this will trigger for both when an item is added or when it’s modified in the list.<\/p>\n Next, you will need to search for the item in the destination list, to get its ID to update it. Although Microsoft Flow does have a\u00a0Filter<\/strong>\u00a0action, you should\u00a0not<\/em>\u00a0use that in this scenario because that will download all of the list items from SharePoint, which will be slow, use up your quota, and simply will not work if the list has more than 256 items.<\/p>\n Instead, you should use the\u00a0Filter Query<\/strong>\u00a0field on the\u00a0Get items<\/strong>\u00a0step in SharePoint. Add the Get items action and select the\u00a0Show advanced options<\/strong> to show all the fields.\u00a0If you are matching the Titles of the rows of the source then you want to type: Title eq ‘[select the title from dynamic content]’ …\u00a0<\/em>(be sure to include the single quote characters):<\/p>\n <\/p>\n You can see the example above. You don’t necessarily need a strict 1:1 column mapping between the two data sources, for example, if you have columns for First Name and Last Name in the source list, and a column for Full name in the destination list, you could type:\u00a0FullName eq ‘[First name] [Last name]’<\/em><\/p>\n When you get the items from the destination list, there are two possibilities:<\/p>\n You will use a condition to tell the difference between these two. Select\u00a0New step<\/strong>\u00a0and then\u00a0Add a condition.<\/strong>\u00a0On the condition card, underneath the left hand box, select Add dynamic content<\/strong>. You will find the collection that contains the list of all the items returned from Get items, it is called value<\/b>. Be sure to select the value from Get items<\/b> (not <\/em>from the trigger When an existing item is modified).<\/p>\n <\/p>\n In the relationship select\u00a0is equal to<\/strong>, and in the value type 0. Now, you will need to go to advanced mode and add the length function. This is important because the value dynamic content returns back the list of items — and you want to check the\u00a0length<\/em>\u00a0of the list and check if that equals 0 (not if the whole list is 0).<\/p>\n In the advanced mode, type <\/p>\n In the\u00a0If Yes<\/strong>\u00a0branch, you will add in SharePoint’s\u00a0Create item<\/strong>\u00a0step. Select the site and list you use above in Get items. Inside the Create item, you should populate each of the columns with fields from the trigger only. You should\u00a0not<\/em>\u00a0use any data from the Get items step because that would be from the destination list (not the source list). Get items will appear above the trigger, so be sure to scroll to the bottom.<\/p>\n In the\u00a0If No<\/strong>\u00a0branch, you will add SharePoint’s\u00a0Update item<\/strong>\u00a0step. Again, select the site and list. Next, you will need to select the Id returned from the\u00a0Get items\u00a0<\/em><\/strong>step.\u00a0<\/p>\n <\/p>\n Note that when you add the ID, you will get an Apply to each<\/em> container automatically added around the Update item<\/strong> step. This is expected — if the query you used in the Get items step is accurate, it will update only the item you wanted to copy. After you fill out the remaining fields (again, be sure to use the outputs from the trigger, not from the Get items call), your condition block will look like this:<\/p>\n <\/p>\n This flow will ensure that whenever an item is added or updated in the first list, that update will be reflected in the second. One limitation with this process is if items are deleted from the first list they will not be deleted from the second. The reason for this is there is no trigger for\u00a0When an item is deleted<\/strong>, thus, there is no way for a flow to be notified when a deletion occurs. Instead, we would recommend that instead of deleting items from your SharePoint lists (or SQL tables, or other data source), you add a column that indicates that the item is no longer needed or relevant. That column will sync between the two lists.<\/p>\n Another limitation is if the field that you are using to keep items in sync between the two lists changes, a new item will be created in the destination list. For example, if you are using a person’s Name to copy between the two lists, and their Name changes, the flow will think that it is a new item, not an update to an existing item. If you can guarantee that the Name will never change then you won’t be impacted by this. However, if Name is a field that can change, you will need to add a new column to the destination list that stores the ID of the item in the first list. In this case, rather than using Name to find items, you can use ID instead (and the ID is guaranteed to always be unique).<\/p>\n Finally, as you may have noticed, this is not a two-way sync. That means that if items are updated in the destination list, those changes will not be reflected in the source list. You should not<\/em> attempt to set up two-way sync with the above flow because that will result in an infinite loop without additional modifications (e.g. list A will update list B, which will update list A, which will update list B, etc…).<\/p>\n The above limitations aside, this is a relatively easy way to have one list always be updated with changes from another list. I hope that you found this helpful, please ask questions in the community<\/a> if you run in to any issues with this type of a flow.<\/p>\n","protected":false},"excerpt":{"rendered":" One common scenario for Microsoft Flow is copying data between two different data sources. For example, if you have a SharePoint list in one site, and you want all of the items to be copied over to a second SharePoint site, you can build flows to do that. This post will walk you through how to do this efficiently.<\/p>\n","protected":false},"author":347,"featured_media":0,"comment_status":"open","ping_status":"open","template":"","power-automate-category":[2836],"power-automate-tag":[],"coauthors":[2913],"class_list":["post-110804","power-automate","type-power-automate","status-publish","hentry","power-automate-category-walkthroughs"],"yoast_head":"\nSetting up the trigger<\/h2>\n
Finding the item in the destination<\/h2>\n
Adding a condition to check if the item exists<\/h2>\n
\n
length()<\/code> around the
body('Get_items')?['value']<\/code> expression. It will look like:<\/p>\n
Create the item<\/h2>\n
Update the item<\/h2>\n
Other limitations and workarounds<\/h2>\n