The Untold Truths About Data Migration Failures

Editor’s note: This article is a guest content contribution from CuroGens CEO, Jesper Kehlet.

Data migrations are super critical to the overall implementation of a new Microsoft Dynamics 365 Finance & Operations (Supply Chain Management) ERP solution. Of course, they are — we want the old data in the new system in some form or fashion, don’t we?

The problem with data migrations as they are being sold to companies implementing ERP solutions with the perception that it is something that is easy, but it’s all about managing expectations and they never exactly spell out what the expectation is.

For example, we saw a client recently being presented with a proposal by a high-end ERP implementation provider, where data migration was two pages out of an SOW of almost 90 pages.

Of those two pages, there was a table of 20 data entities, so the actual description of the data migration was less than half of a page and essentially said, “Load the data that is specified in the following table into one legal entity three times to test the data migration load process and scripts.” Later, it stated, “All data loads thereafter are to be performed by the client as required for all subsequent testing activities.”

If we break that down, it’s essentially not doing anything for a client with more than 100 data entities and 20+ legal entities. But how can a C-level executive even begin to understand that unless some very skilled technical people explained it in detail? The SOW was signed and that’s when the problems started.

The data migration that was presented as simple in the context is a $2.5M+ effort on top of a $5M+ project. The problem? Nobody told the company what to expect, so they believed that the $5M contract value was the total. $7M later, the project is still not done and is facing another year of work.

Data Migrations Deserve Intense Focus and Planning

Data migration takes a lot of planning. Period.

An ERP solution is essentially a relational database system underneath with a lot of tables of data. Something as simple as an invoice may have 30+ tables connected together from the customer record to the invoiced sales order line. A production order involves the same amount of tables. That’s why we need to understand the what, the how, and the where:

• What data are we extracting?
• How should we transform it for the new solution to use it properly?
• Where do we need to put it?

The image below represents Extract, Transform, and Load (ETL) for a data migration project – of which the Transform step is the most critical.

• Maybe the target system needs the data in a different way.
• Maybe it needs components that do not exist in the source system.
• Or the epitome of complexities: Maybe the company wants to see the data in a different way in the new system.

Expect Unpredictability

Data migrations are never really straightforward. And once you get into the detail, you realize that all you know for certain is that there is no real certainty. As you work through the data entities, maybe you find out that there are dependencies that you didn’t think of. Maybe the new system behaves in a different way because new settings were introduced that you didn’t think of.

For example, when a client chooses to implement a different engineering solution to replace the previous engineering solution, you realize that there are things that work substantially different because the new engineering solution vendor treats things differently.

Sometimes a client doesn’t have all the new third-party solutions selected when the project starts and that can throw things off quickly as you may have to go back to the beginning if certain solutions depend on changes to base data.

No matter what you do, there will be unpredictability — and that’s alright, you just need to be prepared to handle it. Have a mitigation plan.

There’s No “I” in Team

One of the most difficult situations you can end up in (and we recently did) is if there are multiple parties involved with different interests.

In the case of the SOW that I mentioned in the beginning, the client already committed to paying the ERP implementation provider for the effort that they estimated and included. It did not make any sense, but we had to work with it. The problem with that is that the ERP implementation provider wanted to take it all over, but since we had a better handle on data migration than they had, the client would not let them – which created a lot of friction.

It took a lot of political maneuvering and change management efforts before we finally got to a proper work structure with the client and the other implementation provider.

But it doesn’t stop there; clients are typically understaffed or don’t have the skills needed to handle the project. Why would they? They need to manage their business, not manage the data migration. However, we need the client as they understand their data best. We don’t know why they chose one value over another 18 years ago — so we need to make sure that they are educated and working with us in a cohesive fashion.

Never underestimate the value of team building. As an old African proverb says: If you want to go fast, go alone, but if you want to go far, go together!

Technical Expertise is Essential

In the field of medicine, there are two primary types of heart doctors: Electrophysiologists and cardiologists — informally known as plumbers and electricians. The plumbers take care of heart valves, blood vessels that are clogged, etc. This is the field where you see bypass surgeries or stents. Then you have the electricians dealing with arrhythmia issues caused by electrical impulses that are misfiring causing ventricular fibrillation and sometimes cardiac arrests without warning. This is where you see pacemakers and ICDs.

My point is, if you have a clogged artery, you wouldn’t go to an electrophysiologist as he wouldn’t be able to treat you properly. Why? It is not his field of expertise.

So, when it comes to moving your crucial organizational data that your compliance standing depends upon, why would you go to anyone but experts in the field?

At CuroGens, we may not be specialists in everything, but the technical expertise to migrate data correctly is our field. Data cleansing, mapping, security, transformation, testing — you are in the right place.

In other words: Don’t let the bicycle mechanic work on your car. We specialize in this for a reason, so come to the experts and get it done right the first time.

---