What a QA Agent Should Actually Check

A QA agent should check the things that are tedious, repetitive, and have a knowable right answer: does every link resolve, does every form validate, does the output match the spec, did anything regress from last time. It should not be trusted to decide whether something is good, only whether it is correct against a defined standard. QA has two halves, conformance and judgment, and a QA agent owns the first half completely and the second half not at all. Confusing the two is how teams get burned.

Check conformance, not taste

The reliable job for a QA agent is conformance: comparing what was produced against an explicit standard. Did the deliverable include every required section. Do the numbers add up. Are there broken links, missing fields, formatting that violates the style guide. These have right answers, and an agent checks them faster and more consistently than a tired human on the tenth review of the day.

Consistency is the real value here. A human reviewer's attention degrades across a long queue; the agent's does not. The hundredth item gets the same scrutiny as the first. For repetitive checks across many deliverables, that alone justifies the agent, the way I argued in running QA across client deliverables.

What the agent cannot do is judge quality. It can confirm a piece of writing has the required word count and no broken links. It cannot tell you the argument is weak or the tone is wrong. That is judgment, and it stays with a person.

Give it a spec, or it checks nothing useful

A QA agent with no standard to check against will invent one, and its invented standard is not yours. The input that makes it work is an explicit definition of correct: the spec, the checklist, the acceptance criteria. Without that, the agent grades against a vague sense of "looks fine," which is worthless.

This is the same principle as a definition of done for AI-assisted tasks. Write down what "passing" means, in checkable terms, and the QA agent enforces it every time. The clearer the standard, the more the agent can own without a human second-guessing it.

For code specifically, the QA agent's job is closer to structured review than to taste, and I split those roles in reviewing an AI-generated pull request: the agent checks the mechanical criteria, the human checks the design decisions.

Make it catch regressions, not just first-time errors

One of the highest-value things a QA agent does is notice when something that used to work stopped working. Humans are bad at this because they do not remember the previous state precisely. An agent can compare against a known-good baseline and flag exactly what changed.

Point the agent at that comparison. Every run, check the current output against the last accepted one and surface the diffs. Most of them will be intended changes; the point is that nothing changes silently. A regression caught by the QA agent before delivery is far cheaper than one caught by the client after.

Flag, do not fix, and log what it found

A QA agent should raise issues, not silently correct them. If it "fixes" problems on its own, you lose visibility into what was wrong and you trust an unreviewed change. Better: it flags, a human decides. That keeps a person in the loop on anything that is actually a judgment call, which is standard human in the loop practice.

And keep a record of what the agent checked and found. When a defect ships anyway, you want to know whether the QA agent missed it or was never pointed at it. A log of checks run and results is the difference between improving the agent and guessing.

The division of labor that works

QA agent owns conformance: links, validation, spec-matching, regressions, the repetitive mechanical checks across every deliverable. Human owns judgment: is this actually good, is the argument sound, is the tone right. The agent makes the human's review faster by clearing all the mechanical noise first, so the person spends their attention on the part that needs a brain.

The prebuilt QA agents at ServoAgent are built to check against an explicit standard and flag rather than fix, which is the only version I trust near real work. Give the agent a spec and the boring checks. Keep the taste for yourself.