A Checklist to Audit an AI Vendor's Governance

Before you depend on an AI vendor, run their governance through a fixed checklist and make them prove each line, not describe it. The rule for the whole exercise is one sentence: if they cannot demonstrate it live, it does not exist. Governance is the one area where a good vendor and a dangerous one look identical on a slide, so you replace the slide with a demand for evidence. Here is the checklist I use before I let any AI system touch anything that matters in my portfolio.

Can they show a real audit trail on demand?

Start here because it is the most predictive line on the list. Ask them to open the record for one real decision the system made and walk you through it: the input, the model version, the reasoning, the output, and any human involvement, all timestamped.

• A serious vendor pulls it up in seconds and the record is structured.
• A weak vendor stalls, offers a screenshot later, or shows an unstructured blob.

The reason this test works is that you cannot fake a good audit trail on the spot, and you cannot build one after the fact. Either the discipline of adding audit trails to the AI system was there from the start or it was not, and this question exposes which in about a minute.

Can they prove every claim they made?

Take their marketing page and turn each claim into a request. They say accurate: show me the accuracy measurement and how it is tracked. They say secure: show me the access controls and the log of who touched what. They say monitored: show me the monitoring.

The point is not to be hostile. The point is that claims discipline is a proxy for engineering culture. A vendor who only makes claims they can demonstrate has built a certain kind of company. A vendor whose claims dissolve under a simple "show me" has built a different kind, and you will meet that difference again at the worst possible moment.

Does the system enforce its own limits?

Ask what happens when the system is asked to do something outside its authorized scope. A governed product refuses and logs the attempt. An ungoverned one tries, because its limits live in a policy document rather than in the code.

This is the difference between guardrails you can point to and guardrails you can only hope for. Push until you understand whether the boundaries are enforced at runtime or merely written down somewhere. Enforced limits are the foundation of the reliability enterprise buyers actually need proven, and they are the thing most demos quietly skip.

Are human overrides recorded?

If the system has a human review step, ask exactly what gets stored when a human approves, rejects, or overrides the AI. You want a reason, an identity, and a timestamp captured every time.

An override that leaves no trace is not a control, it is a gesture. Weak vendors add a human to feel governed and forget to log what the human did, which means the single most important events in the system are the ones with no record. Make them show you a real recorded override, not a description of one.

Do you keep control of your own data and evidence?

The last section is about leverage. When you need the audit records for a specific customer on a specific date, how do you get them? Can you export everything if you leave? Who at the vendor can read, or edit, your logs?

A vendor who makes you file a support ticket to see your own decision history will make you wait when you can least afford it. And a log the vendor can silently edit is a story, not evidence. Insist on append-only records, self-serve retrieval, and full export. Ownership of the evidence is part of governance, not a nice-to-have.

How to score it

Nine questions, one rule: proven or it does not count. Tally how many the vendor demonstrated live versus described.

• Mostly demonstrated: this is a company that treats governance as a product surface. Proceed.
• Mostly described: this is a company betting it can add governance later if forced. In a regulated context, walk.

I run exactly this checklist on my own products before I ship them, and I build them at Girard AI so every line has a live answer before a buyer asks. Governance is not a document you request from a vendor. It is a set of behaviors you make them perform in front of you, and this checklist is how you make them perform.