Pin the Model Version in Production, Don't Float It

Pin the exact model version your product runs on, and upgrade it deliberately, never automatically. If you call a floating alias that always points at the provider's latest model, you have handed control of your product's behavior to someone else's release schedule. One morning the provider ships an update, your outputs shift, your eval scores move, a feature that worked starts failing, and you have no idea why because nothing in your codebase changed. Pinning is the boring discipline that keeps an AI-native product from silently breaking underneath you.

The thesis: the model version is a dependency like any other, and floating a dependency in production is how you get surprised. Pin it, test upgrades against your evals, then move on your schedule.

Why a floating model version is a liability

In deterministic software you would never point production at "latest" of a critical library without testing. You pin versions and bump them on purpose. A model is a bigger dependency than most libraries, because it drives the core behavior of the feature, and it is one you do not control and cannot inspect.

When the provider updates a floating alias, the change lands with no diff, no changelog you can act on, and no warning. Your carefully tuned prompt was tuned against the old behavior. The new model may be better on average and worse on your specific cases, and you find out from users. This is the mechanism behind how model upgrades erase capability edges: the ground moves and you were standing on it.

How to pin and upgrade safely

Pin the specific version string in config, not a floating alias. Now your behavior only changes when you change it, which is the whole point.

Gate every upgrade on your eval set. When a new model version ships, run it against your golden eval set before it touches production. A higher benchmark from the provider is not evidence it is better for your use case, your evals are. If the score drops on your cases, you do not ship it, no matter how impressive the launch post was.

Re-tune the prompt against the new version if you adopt it. Prompts are fitted to a model. A prompt that was optimal on the old version may be mediocre on the new one, so treat an upgrade as a change that needs its own tuning and its own offline eval pass, not a drop-in swap.

Keep the ability to roll back. If a new version misbehaves in production despite passing evals, you want to flip the config back to the pinned old version in seconds. That is only possible if you pinned in the first place.

The cost of pinning, honestly

The caveat, because pinning is not free. Pinned versions get deprecated. Providers retire old models on their timeline, and if you pin and ignore it, you will eventually be forced to migrate on short notice, which is worse than migrating on your own schedule. So pinning is not "set and forget," it is "set and track." Watch deprecation notices and plan upgrades before they become emergencies.

Pinning also means you do not automatically get improvements. A better model sits available while you run the old one until you do the eval work to adopt it. That is a real tradeoff, and it is the right one, because an improvement you did not verify against your cases is a gamble, not a gift. The reliability of the feature depends on you controlling when its core behavior changes.

The rule I hold products to

Every model-backed feature in the Girard AI portfolio runs on a pinned version, upgrades only after passing the eval gate, and keeps a rollback path. Floating to latest feels convenient right up until the morning it quietly breaks your product with no code change to blame. Control the one dependency that controls your behavior. Pin the model, test the upgrade, move on your own schedule.