How to use Power Automate to Import & Deduplicate data from Excel Online to the Dataverse

How to use power Automate to import, deduplicate and update records stored in an Excel Online file to the Dataverse.

We know that importing contact records from an Excel file to the Dataverse is fairly easy, there are several tools available, however, have you tried to import records from a Excel Online file using Flow? I recently needed to do this and to my surprise, it was not as simple as I originally thought.

In this post we will review some of tools and functions I used to create the final solution.

  • How to use the Power Automate Search records action (Currently in preview)
  • How to use the empty() expression
  • How to use a condition to check if an array is empty
  • How to import records from an Excel Online file using Power Automate

Recently, I needed to import contact records that were in a OneDrive folder, however, about 50% of the contacts had already been incorrectly uploaded, so I need to not only import the records but check for duplicates and fix the existing records.

In short, I needed to create a Flow that will search for a contact record in the Dataverse using an email address, if the record is found, clear out the first name, update the last name and set the contact type. If the contact was not found, then create the contact record.

How hard can this possibly be? Well, it was not as straight forward as I thought.

How to deduplicate and import records from Excel Online using Power Automate (Flow)

Before we jump right into how to create the Flow, lets evaluate the requirements.

Current Application

  • Dynamics 365 Marketing

Requirements

  • Import a list of contacts from an Excel Online file
  • 50% of the contacts have already been imported but in error, their first name and email address is the same value, this must be corrected.
  • For any contacts that have not be imported, import them.
  • Update all the contacts to have a contact type of Account.
  • Create a segment in Dynamics 365 Marketing, for all contacts of type Account.

Some challenges I faced while creating the flow

  • In order to use the Get row by Id action, I needed the Dataverse record ID, so that rules out using this action.
  • I tried using the List Rows action and filtered with the email address from the Excel file using FetchXML but I since you cannot access the dynamic values in the filter, that did not work. I know, sounds weird, but unless there is a trigger or an action before the list record step, their Dataverse columns are not available. I was using a PowerApps Trigger.
  • Since I could not find the Rows using the Get Row by ID, I had a look at what other options were available, and to my surprise, there is a new action called Search rows (preview). Fantastic, just when I needed it.
  • Final challenge I faced was, trying to use a condition to determine if any records were found from the search. When a record is found, the output is an Array of Objects otherwise an empty array. So, you would think that I could just test the length of the array using the expression length(array) = 0 to determine which records are found and not, well Flow does not allow this.

So, here is how to configure the Flow to deduplicate, import and update records in one go.

To start, create a blank Instant Flow

Add the PowerApps trigger since this Flow will run on demand

Add a PowerApps V2 Trigger
Add a PowerApps trigger

Next, add the List rows present in table action, and set the location of the file as per your file configuration. (Your data must be in a table format).

Add a list rows action
This action will retrieve the records from the Excel Online file

Add an Apply to each control and set the first parameter as the output value from the list rows action

Add an apply to each control
The Apply to each control is a loop that will process each of the records in the Excel file

Next, add the Search rows action and set the search item to the dynamic value of the email address in the excel file, then set the table filter item to the logical name for the contact table (contact).

Add the search rows action
The Search rows action uses the email address from the Excel file to find a matching record in the Dataverse.

Next, you should add a condition control, this will be to test the result of the search to determine if a record was found with the matching email address.

Use the empty() expression to test the output
The condition will test if a record is found. If no record is found an empty array is returned. empty(outputs(‘Search_rows_(preview)’)?[‘body/value’])

Next, on the if yes branch, add the Dataverse Add a new row action and map the fields from the Excel file to the Dataverse columns.

Map the Excel columns to the Dataverse columns
Map the Excel data columns to the Dataverse columns.

On the If no branch, add an apply to each action and set the first parameter to the output of the List rows.

Add a get row by id action
If the record is found, you will retrieve the record and then update it with the values from the Excel file.

Finally, add a Dataverse Update a row action and map the columns you would like to update

Map the Excel online columns to the Dataverse columns
Map the Excel columns to the Dataverse columns you would like to update. To clear out any values, use the null expression

There you have it, the Flow will retrieve the contacts from the Excel Online file, search for them in the Dataverse using the email address, if the record is not found, it will be created, otherwise, the record is updated as needed.

Once you have complete configuring the Flow, it should look like this.

The complete version of the flow

I hope you found this article helpful.

Until next time, happy CRMing!

Byron Dittberner, Dynamics 365 & Power Platform Solution Specialist
Byron Dittberner | Dynamics 365 & Power Platform Solution Architect

5 comments

  1. Nitish Kurgode - Reply

    Hi Byron,

    Hope you are doing well!

    Thanks for making this detailed blog. It’s really easy to understand for someone like me who is just getting started with power automate.

    My update a row function is creating new records in dataverse. I’m unable to understand where I’m going wrong. I followed all the steps from this blog. I’m doubtful about the values I have used in my Get a row by ID function as it wasn’t mentioned what exact values to use in it.
    Could you please provide information about the table name and the row ID you used in Get a row by ID function?
    mine are as follows:
    table name: contacts
    Row ID: list of rows Row object ID

    Is that correct?

    I read in forums that I need to pass a GUID value and If that is not present, update a row function can insert new records. Do I have to do that?

    Your help is sincerely appreciated
    Best regards and thanks,
    Nitish

    • Byron - Reply

      Hi Nitish, you are on the right track, Row object ID should be correct. Remember, the ID will only be returned if a record is found otherwise you will be passing an empty ID into the Get row by ID action.

  2. Bianca - Reply

    Not all heroes wear a cape!

    This solution worked perfectly for me. I have had this issue for over a week and I just came across your post and I am so glad I did!

    I had the same issue as this previous comment and I would like to share how I fixed it: for both Get row by ID and Update row by ID – I used “List of rows Row object ID” because I am using dynamically an Excel that can be imported thru PowerApps and needs to update Dataverse.

    Thank you, sir, for sharing this wonderful solution!

  3. Sam Conroy - Reply

    This works great but I seem to be restricted to 256 results on the Apply to each step. Have I missed something?

    • Byron - Reply

      Hi Sam

      I assume the limitation you are referring to is the number of rows in your Excel file, if so, I would suggest you create a table in you Dataverse to store the records temporarily (staging table) then once the import has completed, process the record in the staging table by running the deduplication Flow. Once the process is complete, add a delete action to the last step of your flow to delete the records in the staging table. This approach would require several changes to your Flow but is should be rather simple.

Leave a Reply

Discover more from Dynamics 365 & PowerApps Blog

Subscribe now to keep reading and get access to the full archive.

Continue reading