How to Plan a Data Migration

Who This Guide Is For

Business owners and project managers who are moving from one system to another — replacing a legacy tool, consolidating platforms, or migrating to a custom-built system — and need to ensure their data arrives intact.

Before You Start

• Data migration is a project in itself. It is not a side task within a software development project. It needs its own plan, timeline, and testing.
• Your data is messier than you think. Every legacy system accumulates inconsistencies, duplicates, and gaps over time. Discovering these early is essential.
• Plan for rollback. If something goes wrong during migration, you need a way to revert. Never destroy the source data until the migration is verified.

Step 1: Audit Your Current Data

Before moving anything, understand what you have. Export your current data and examine it:

• Volume. How many records? How large are the files?
• Structure. What fields exist? What data types? Are there relationships between tables?
• Quality. How many duplicates? How many incomplete records? Are formats consistent (dates, phone numbers, addresses)?
• Dependencies. Does other software or reporting rely on this data in its current form?

This audit almost always reveals surprises. Better to find them during planning than during migration.

Step 2: Map Source to Destination

Create a field-by-field mapping between the old system and the new one. For each field:

• Direct mapping. The field exists in both systems with the same format. Copy directly.
• Transformation required. The field exists in both but the format differs (e.g., dates stored as text vs. dates stored as dates). Define the transformation.
• No equivalent. The field exists in the source but not in the destination. Decide whether to add it, archive it, or discard it.
• New fields. The destination has fields that do not exist in the source. Define default values or leave them empty for manual population.

Document every mapping decision. This document becomes the specification for the migration scripts.

Step 3: Clean Before You Migrate

Migration is an opportunity to improve your data quality. Before moving:

• Deduplicate. Merge or remove duplicate records. Define rules for which record takes precedence when duplicates conflict.
• Standardise formats. Normalise phone numbers, addresses, and dates to consistent formats.
• Fill gaps. Where practical, complete missing fields. Where not, flag them for post-migration attention.
• Archive dead data. Records that are no longer relevant (closed accounts, obsolete products) can be archived rather than migrated.

Cleaning data in the source system is easier than cleaning it after migration.

Step 4: Build and Test the Migration

Write the migration scripts or configure the migration tool, then test rigorously:

• Run on a subset first. Migrate 100 records and verify them manually. Check every field, every relationship, every edge case.
• Run on the full dataset. Migrate everything to a test environment. Compare record counts. Spot-check random records. Verify totals and aggregates match.
• Test downstream systems. If other tools or reports depend on this data, verify they still work with the migrated data.
• Test with users. Have the people who use the data daily review the migrated records. They will catch issues that automated checks miss.

Step 5: Execute and Verify

When you are confident in the test results, plan the production migration:

• Schedule downtime if needed. If both systems cannot run simultaneously, communicate the outage to affected users.
• Take backups. Backup the source system immediately before migration. Backup the destination system after migration.
• Run validation checks. After migration, verify record counts, key relationships, and data integrity.
• Monitor for issues. The first week after migration is critical. Watch for missing records, broken references, and user-reported problems.

Common Mistakes

• Underestimating the timeline. Data migration typically takes two to four times longer than initial estimates. Build buffer into your schedule.
• Skipping the data audit. Migrating dirty data to a clean system just creates a clean system with dirty data. Clean first.
• Not testing enough. One test run is not sufficient. Test multiple times, with different data subsets, and verify results each time.
• Destroying the source prematurely. Keep the old system accessible (even read-only) for at least 90 days after migration. You will need to reference it.
• Treating migration as a one-time event. Some data continues to flow into the old system during transition. Plan for delta migrations to capture changes made between your test migration and go-live.

What Good Looks Like

A successful data migration results in all records accounted for, no data loss, consistent formatting, and users who trust the new system’s data from day one. The old system remains accessible as a reference, and there is a documented audit trail of every migration decision.

Next Steps

If your data migration is part of a larger system replacement, see How to Plan a Legacy System Replacement. For migrations tied to custom software development, Technical Consulting can help scope the data work alongside the build.