Write Characterization Tests Before You Let AI Refactor

Before you let an AI refactor code that has no tests, write characterization tests. A characterization test does not check that the code is correct. It checks that the code still does exactly what it did before. That distinction is the whole point. Refactoring means preserving behavior, and you cannot preserve what you have not captured. Without these tests, you have no way to tell a clean refactor from a silent regression, and neither does the agent.

This matters more with AI than with human refactoring, because an agent refactors fast and broadly. It will touch more code in one pass than a person would, which means more surface area for behavior to drift. A characterization test suite is the net under that fast, broad change. It turns red the instant the agent alters an output, even an output nobody remembered the code produced.

What a characterization test actually is

It is a test that records current behavior as the expected behavior, correct or not. You run the existing code against a set of inputs, capture whatever it returns, and assert that future runs return the same thing. You are not judging whether the output is right. You are freezing it so you can detect any change.

This feels strange the first time, because you might be locking in behavior you suspect is wrong. That is fine. The refactor is not the time to fix bugs; it is the time to restructure without changing behavior. If the old code had a quirk, the refactored code should have the same quirk, and the test proves it does. Fixing the quirk is a separate task with its own test, done after the refactor lands. Mixing the two is one of the AI-assisted development mistakes that makes a regression impossible to isolate.

Why the agent needs this net, not just you

You might think the net is only for your peace of mind. It is also a signal the agent can use. When characterization tests exist, the refactor loop has an objective pass-fail: green means behavior held, red means it moved. The agent can run the suite after each step and know whether it stayed inside the lines. Without the suite, the agent is guessing that its change preserved behavior, and it guesses optimistically, same as it does everywhere else.

This is what lets you refactor in the small, safe steps that actually work, which I lay out in refactor with AI in small steps. Each step ends with the suite. If it is green, commit and move on. If it is red, the agent overstepped, and you catch it immediately instead of three steps later when it is buried. The net is what makes small steps meaningful.

Let AI help write the characterization tests

Here is a useful trick: agents are good at generating characterization tests, because the job is mechanical. Point the agent at a function, have it enumerate representative inputs, run them, and capture the outputs as assertions. It will produce broad coverage faster than you would by hand, and the circularity problem that plagues agent-written correctness tests does not apply here, because these tests are not asserting correctness. They are asserting sameness, and sameness is exactly what a machine can capture without judgment.

Do sanity-check the captured outputs before you trust them as the baseline, in case the code is currently crashing or returning something obviously broken on an input. But the bulk of the work, running the inputs and recording the results, is ideal to hand off. This is the kind of tedious, high-volume task where AI earns its keep and a human would cut corners out of boredom.

Then refactor with confidence

With the suite in place, the refactor becomes low-stress. The agent restructures, the suite confirms behavior held, and you review the diff for whether the structure genuinely improved. Two independent signals: tests for behavior, eyes for structure. That is a far better position than staring at a big diff and hoping nothing shifted.

Understanding what a change will touch before you start makes this even safer, which is where repo-level tooling comes in; I run refactors through repo intelligence before you refactor to see the blast radius, and through platforms built for the loop, like Bootspring for the build cycle and ReformCode for reading the codebase you are about to change. Write the net first. It is the cheapest insurance in AI-assisted development, and skipping it is how a routine refactor turns into a production incident nobody can explain.