Harnessing Quality in the AI Era

>_You own the outcome, not the machine

You ship a feature powered by a model you did not train and cannot inspect. The provider can swap the weights behind your endpoint on a Tuesday, and your product behaves differently by Wednesday. No changelog arrives. And when a customer screenshots a confident, fluent mistake, the escalation lands on your desk, not the provider's. That is the bargain of building on foundation models: total accountability, partial control.

The instinct is to call this someone else's problem, the model's or the provider's. It is not. Quality was never a property of the model; it is a property of the system you build around it, and that system is entirely yours. Harnessing quality in the AI era means putting enough structure around an unpredictable core that you can still promise something to a user and keep it. What follows is the framework I have landed on after years of owning quality on both deterministic and probabilistic systems.

>_Define quality before you measure it

Every quality effort that fails, fails here first. The team jumps to dashboards and eval scripts before anyone has written down what 'good' means for their product. Quality is not generic: a legal summarizer and a casual chatbot fail in different ways, and a metric that matters for one is noise for the other. If you cannot state your quality bar in a sentence a new engineer would understand, every review becomes an argument about taste.

So write it down, per domain, as named dimensions with a target on each, concrete enough that two people scoring the same output would mostly agree. This document is the spine of everything downstream: evals grade against it, guardrails enforce its hard limits, and humans review what is ambiguous.

• -Faithfulness: does the output stay true to the source, or invent facts, citations, and policies that were never there.
• -Task success: did it do the job the user actually asked, not just produce plausible text near the job.
• -Safety and tone: is it on-brand, non-toxic, and unwilling to be talked into things it should refuse.
• -Format: does it return valid JSON, respect the schema, and stay inside length and field limits.
• -Cost and latency: a perfect answer that costs a dollar and takes nine seconds can still be the wrong answer.

>_Instrument everything, then instrument more

You cannot manage what you cannot see, and most AI systems are near-blind by default. A request goes in, an answer comes out, and the interesting part, what the model was actually told and why it responded that way, evaporates. Before you write a single eval or add a guardrail, fix the visibility: instrumentation is the foundation the rest of the framework stands on.

Trace every request end to end and store it: the resolved prompt, the retrieved context, the model version, the parameters, the raw output, token counts, latency, and cost. Attach a stable id so you can follow one interaction across logs, evals, and incident reviews. When something breaks, the gap between a five-minute diagnosis and a two-day guessing game is whether you captured the exact inputs that produced it, since the model may already have changed underneath you.

>_Evals as contracts

An eval is more than a test. It is a written contract that states what good means and where the passing bar sits, and holds the model to it on every change. Pin that contract in the repo next to your code and 'the summaries got worse' stops being a hallway complaint and becomes a number that moved, with the offending cases attached. I go deep on the machinery in /resources/what-are-ai-evals; here I care about the role it plays.

Treat the eval as the gate no change passes without facing. Every prompt tweak, model swap, or retrieval change runs the dataset, scores against your dimensions, and reports per slice so a win on easy cases cannot hide a regression on hard ones. Offline in CI it is your regression contract; sampled on live traffic it is your early-warning contract, telling you faithfulness slipped three weeks running before your churn numbers do. A contract nobody enforces is decoration, so wire it in and let it block.

>_Wrap the model in guardrails

Evals tell you how the system behaved yesterday, across a sample. Guardrails decide what a single user sees right now, when the model does something it should not. One is statistical and after the fact, the other deterministic and in the path. You need both: some failures are attacks, and some outputs are unacceptable no matter how rare. The rule: treat the model as untrusted, and put deterministic code on both sides of it.

• -Validate the input before it reaches the model: strip or sandbox untrusted content, enforce size limits, and never let user-pasted text issue instructions to your system.
• -Validate the output before it reaches the user: schema checks, allow-lists, banned-phrase filters, and range checks on any number the model produced.
• -Fail to a safe default: when an output flunks a hard check, fall back to a canned response or a human handoff rather than ship a confident mistake.
• -Keep a kill switch: a deterministic way to route around the feature the moment a class of failure starts trending, with no deploy required.

>_Put humans where they change the outcome

'Human in the loop' gets said like a virtue, as if more review were always safer. It is not. A human on every request is slow, expensive, and numbing, and a reviewer rubber-stamping the ten-thousandth low-risk answer catches nothing. The framework treats human attention as your scarcest resource and spends it only where a wrong answer costs more than a human minute.

That means tiering by stakes. Let the model act on its own for low-risk, reversible actions your guardrails already cover. Insert a human approval step where an error is expensive or hard to undo: sending money, deleting data, making a legal or medical claim, anything a customer would screenshot. Route the genuinely ambiguous middle to a person whose judgment feeds straight back into the eval set. Human review is not only a safety net; it is your richest source of new test cases.

>_Manage the model like the dependency it is

You would never let a third-party library silently auto-upgrade in production, yet teams do exactly that with the most consequential dependency in the stack. The foundation model is a dependency: external, versioned, and able to change your product's behavior without touching a line of your code. Manage it with the rigor you give every other dependency and the provider's next update stops being a threat.

• -Pin explicit model versions; never point production at a floating 'latest' alias you do not control.
• -Treat every version bump as a migration: run it against your full eval before it reaches a user, not after.
• -Canary new versions and prompts on a slice of traffic behind your eval and monitoring, and compare score distributions before rolling forward.
• -Keep rollback one switch away and rehearse it before you need it, because you will need it.
• -Hide the provider behind a thin interface so swapping models or falling back to a second one is a config change, not a rewrite.

>_The framework is an operating loop

Written as a list these look like separate practices. In a healthy team they are one loop: you define quality, instrument to see it, encode it as eval contracts, guard its hard edges, spend human judgment where it counts, and manage the model as a dependency. Then every production failure feeds a sharper definition, a new trace, another eval case, one more guardrail, and the loop tightens with each incident instead of fraying. Someone has to own it, hold the bar when a flashy demo scores worse than the boring version it beats, and keep the machinery pointed at production. That is leadership work, and I get into it in /resources/qa-leadership-principles.

You will never control the model, and you do not have to. What you control is whether, at any moment, you can answer one question with evidence instead of hope: is my system getting better or quietly getting worse? Build the framework that answers it, and not owning the model stops being the thing that keeps you up at night.