What to Ask an AI Vendor About Their Audit Trail

Ask the vendor to show you what their system logged for one real decision, end to end, right now on the call. If they can pull it up in ten seconds, you are talking to a serious company. If they stall, promise a screenshot later, or explain that logging is "on the roadmap," you have your answer. The audit trail is the single fastest way to tell a governed product from a clever demo, and most buyers never ask about it.

I build across a portfolio of AI companies, and the ones that win regulated buyers all share one trait: they treat the log as a product surface, not a debugging afterthought. Here is what to actually ask.

Can you show me the audit trail for a single decision?

Not the concept. The artifact. Make them open the record for one action the system took and walk you through it. A real audit trail shows the input, the model or rule that ran, the version of that model, the output, the human who reviewed it if anyone did, and the timestamp on each step. If the record is a blob of raw text with no structure, it will be useless the day you actually need it, which is the day something goes wrong and a regulator or a plaintiff's lawyer asks what happened.

The reason this matters is simple. You cannot reconstruct an audit trail after the fact. Either the system captured the decision when it happened or the evidence does not exist. This is why I argue that you have to add audit trails to AI systems as a foundational layer, not a feature you enable in year two.

What exactly gets logged, and what gets dropped?

Push on coverage. Some vendors log the final output but not the reasoning path. Some log the prompt but not which model version answered it. Some log everything in staging and sample one percent in production to save money. You want to know the gaps before you depend on the system, because the gap is always in the exact place you will need to look.

Ask these specifics:

• Do you capture the model version and configuration for every call?
• Do you log the full input, or a truncated version?
• When a human overrides the AI, is that override recorded with a reason?
• How long is the retention, and can I export it?

The override question is the one that catches people. A human in the loop is worthless as a control if the human's decision vanishes into the void. The override is often the most important event in the whole chain, and it is the one weak systems forget to store.

Is the log tamper-evident?

A log that anyone on the vendor's team can quietly edit is not evidence. It is a story. Ask whether records are append-only, whether they are hashed or signed, and who has write access. In legal and financial contexts this is not paranoia, it is the whole point. The value of the trail is that it can be trusted by someone who does not trust you, which is exactly the situation you are in when a dispute lands.

This is where governance stops being a checkbox and starts being a moat. Any competitor can match a model's raw capability within a release cycle. What is hard to copy is a system built from day one to prove what it did. I have written before that capability is a commodity and governance is the moat, and the tamper-evident log is where that thesis gets concrete.

Who can read the trail, and how fast?

The last question is operational. When your compliance team needs the record for a specific customer on a specific date, how do they get it? Self-serve search? A support ticket with a two-day turnaround? A vendor that makes you beg for your own audit data will make you beg at the worst possible moment.

For regulated buyers, speed of retrieval is part of the control. In personal injury work, for example, the firms that adopt AI case management they can trust do it because they can produce the record on demand, not because the model is a little smarter than the last one.

The short version

The audit trail is the cheapest due diligence you will ever run and the most predictive. Ask to see one real decision. Ask what gets dropped. Ask whether the log can be edited. Ask how fast you can pull it. Four questions, ten minutes, and you will know more about the vendor's engineering culture than any pitch deck could tell you.

I run this test on my own products before I ship them, and I build them at Girard AI so the trail exists before the feature does. A vendor who cannot show you the log is not hiding a small gap. They are telling you governance was never the plan.