← Back to home
Field notes

Migrating a Legacy System Without Breaking It: Notes on Data Sync

You can't just switch off the old system and switch on the new one. That's the part people underestimate every time.

A legacy system has consumers - other tools, other teams, sometimes other companies - that depend on it staying up, no matter how old or how badly written the code underneath is. And writing the replacement as a full clone, business logic and all, before touching anything else, takes a long time. Long enough that if the legacy system keeps receiving new features while you build, you end up building the same features twice - once in the code you're trying to retire, once in the code that's supposed to replace it.

So the real problem isn't "how do we build the new system." It's: for the months this takes, information has to exist in both systems, and both have to agree with each other.

Here's what I've learned building that.


Table of contents

No single technique is enough on its own

There are a handful of standard approaches to this - dual writes, change data capture, event-driven sync. In practice, using any one of them alone almost never works, outside the simplest cases. What actually gets you through a migration without getting stuck halfway is combining them. That combination is the part that doesn't show up in most explanations of these patterns, because each one is usually described in isolation, as if you'd pick exactly one and be done.

You won't. Here's why, one technique at a time, and where each one runs out.

Dual writes

Write to both systems. New record goes into the new system and the old one, at the same time. If one of the writes fails, you sync afterward so you don't end up with two different versions of "the truth."

This is the simplest to reach for, and the sync-after-failure part is where it gets fragile. You need a reliable way to detect that one side failed, and a reliable way to replay it - and if that detection itself has a gap, you get silent divergence that nobody notices until a customer does.

Change Data Capture

Read the changes happening in the old system's data directly, and stream them continuously - every insert, update, delete, turned into an event as it happens. This is more reliable than dual writes because it's reading what actually happened at the data layer, not just what the application thought it was doing.

CDC solves "did the change actually get captured." It doesn't, on its own, solve what happens once you're writing in both directions.

The backward sync problem

Once both systems are live, you're not syncing one direction - you're syncing both. New system write needs to land in the old system. Old system write needs to land in the new one. And this is exactly where it's easy to fall into an infinite loop: the new system writes to the old one, which triggers a sync back to the new system, which looks like a fresh change, which syncs back to the old system again - indefinitely.

The way out is an anti-corruption layer: attach metadata to every change so you know which system it originated from. This shows up most often in event-driven integrations - every integration, old and new, checks the origin before accepting a change, and drops anything that originated with itself. You can also route this through separate channels entirely - in something like RabbitMQ, have the new system publish to one exchange and the old system to another, so they're not even listening to their own echoes.

Authority flags - the piece that actually resolves it

This is the one I'd point to first if I had to pick. It solves the backward-sync problem more directly than origin tags do, because instead of detecting the loop after it happens, it prevents the loop from being possible.

Both systems stay active. Either one can be authoritative for a given piece of data - the point is that only one of them is, at any given moment, for that specific record.

Say you're adding an order. If the new system is authoritative, the new system does the write. That write gets synced to the old system. The old system's readers keep working exactly as before, reading a copy that's kept current - right up until you're ready to flip which system handles writes, at which point the new system takes over and the old system becomes the mirror.

Or the reverse, depending on which side of the migration a given entity is on. Either direction works, and both are practical - the point isn't which one you pick, it's that exactly one of them is true at any given time, for any given record.


Why the combination matters more than any single piece

Dual writes give you a way to keep both systems populated during the overlap. CDC gives you a reliable feed of what actually changed. The anti-corruption layer keeps that feed from turning into a loop. The authority flag is what turns "we're syncing both directions and hoping" into "exactly one system owns this record right now, and we know which."

None of these replace each other. In every migration I've worked through, at least two of these were running simultaneously, layered - CDC feeding the sync, authority flags deciding who's allowed to write, an anti-corruption layer catching anything that slips past the flag. Picking only one and expecting it to carry the whole transition is the mistake that gets teams stuck three weeks in, staring at duplicated records with no clear story for how they got duplicated.


What this buys you, practically

Once authority can move independently - per entity, sometimes per field - the migration stops being one enormous cutover event. It becomes a series of small, reversible moves. Shift authority for one type of record. Watch it run for a week. If something's off, shift it back - nothing is lost, because the other system was only ever a mirror during that window.

That reversibility is the whole point. It's what makes a migration something your team can do incrementally, on a schedule the business can actually live with, instead of a single night where everything either works or it doesn't.

Skybridge Systems works on exactly this kind of migration - old system still running, new system coming online, and the sync between them that has to hold while both are live. Get in touch if you're in the middle of one and want a second opinion on the approach.