There are various tools out there we can use to move configuration data from one Dynamics environment to another, but what if we could add data directly to our solutions? So that there would be no need to run XrmToolBox, package deployer, SSIS, Scribe, or other tools in order to get our base data, always with the same guid-s, deployed? So that we would provide the solution file, and, once the solution has been imported, it would take just one click of a button, all in Dynamics, to get that data imported?
If this sounds remotely interesting, there is a tool I’ve been working on which you might want to try:
You can download it from here:
Here is how it works:
- Using the tool, you can create different datasets. Those datasets are based on FetchXML, and you can give them priority order (When “Import All” option is selected, the dataset with the lowest order # will be deployed first. For example, if you have lookups somewhere, you might want to deploy your “parent” records first followed by the “child” records). Creating a new data set is simple – just click “New Data Set” button:
- Once you have a data set, you can prepare all data or just that particular data set
- This is where there is some difference between “home” datasets (those created in the current Dynamics organization) and external datasets. You cannot update external datasets – you can only load data from them.
- For every dataset, there will be a web resource
Once the datasets are there, and once you have prepared the data (using “Prepare All”, for example), you can include those web resources into a new/existing solution, and, then, bring them over to another environment. As in the example below, I have two web resources included into a separate solution:
I might have included them into an existing solution – it does not matter. What matters is that, once you bring those web resources to the target environment, and if you have Configuration Data tool installed there as well, you can use “import all” option (or import each individual dataset separately) to deploy that data in the target environment:
For example.. I have no accounts here other than “test” account:
I am using “Import All”:
And now I do have the accounts:
As for the limitations.. The main limitation of this first version, is that it does not support N:N system entities and some of the more advanced data types (such as multiselect optionsets, activity parties, and/or customers).
And there is a bunch of improvements I can see right away (including ID mapping, solution mapping, more advanced lookup resolution mechanism, moving the whole thing to GitHub, etc).
Yes, there is some work to do for version 2. For now that’s what it is, though, so let me know what you think!
Updates by version #:
1.0.4: http://www.itaintboring.com/data-migration/configuration-data-manager-version-1-0-4-0/
I haven’t gotten an import to succeed yet with my own data given various errors, but love the idea. Here are a couple initial observations and feedback:
1. The download links don’t work, but I was able to get the right URLs after tweaking it
2. There’s an error when saving a data set when the Fetch XML is more than 2,000 characters
3. Imports fail when lookup records don’t already exist in the target organization
4. The data set web resources have xml file extensions, yet when opening them they aren’t xml, perhaps an encoding issue?
This is very promising, if you’re committed to the cause I encourage you to keep working on this. This is a much more intuitive approach for managing configuration data than the CRM SDK’s configuration migration tool. More sophisticated UIs and complex data handling could easily make this a viable alternative.
Hey Alan, thanks for the feedback, much appreciated.
May have to move fetch to the web resource itself to overcome that 2000 limit..
#4 is more like “internal” details, since the data is compressed anyway. But it may make sense to use a different resource type instead so it is less confusing. Will do it in the update.
#3 is the most challenging part (apparently). This is where right now you can handle it by correctly ordering those datasets (so that referenced data is imported first), but I’ll need to work on it to make this an easier process.
And I’ve fixed the links:))
Hi Alex,
I like your idea and will test your solution.
One comment from my side: Do not forget to delete the webressource in target system after importing the data. 🙂
Hi Lars,
hm.. I was thinking more along the lines of keeping it there. Those web resources are supposed to be part of the managed/unmanaged solution that’s deploying the configuration, so I don’t really want to mess with those. But (and that’s not there yet), there will also be “appliedon” date stored somewhere, and “import all” will have an optional “override data” switch.
This tool looks great and I am keen to try it out – but on my first attempt to move some data, get an error. I’m trying to duplicate my products table from a production environment to sandbox. When I try to import the file, I get an error:
product With Id = {guid} Does Not Exist
i would edit the xml and ensure that child products are created before product families, but i cant edit the XML itself. Any suggestions?
There are certain things in the lookup resolution I may still need to work on, but if you did not get an updated version (it’s 1.0.1.0 in the solution version number), you may want to try that one – just use the same download links. In that version, if a lookup cannot be resolved, it’ll be set to null. Which might not be correct in your case – it depends. But, once you have all the records (including parent records), you can simply re-apply the same data set again. What’s important is that you have to have units with the same “id-s”.. unless your sandbox is a copy of production, those Id-s may be different, and that’s where right now the tool is missing some kind of more intelligent lookup resolution mechanism
What it’s complaining about is, likely, parentproductid field, so you might also try to set up 2 datasets:
– The first one will have a condition in the fetch (Parent does not contain data)
– The second one will either have all of the products or just those where Parent contains data
Then either apply them in that order, or use “import all” but don’t forget to assign correct order to those 2 datasets. But again, that’s assuming there will be no issues with the units and groups.
Hi,
I’m using the ConfigurationData Solution for some months already and its a quite good approach for me deploying configuration data records through the organizations.
One thing I became aware recently. There are plugins running when retrieving webressources.
Question: What is it used for?
I want to avoid any kind of processes running when retrieving data especially when the ConfigurationData only comes into action when deploying changes.
Thank you in advance
Best regards
Lars Martin