Trust is a runtime property

I've been thinking about this through safety classifiers. The standard setup: every piece of user-generated content passes through a classifier before it surfaces, and if the classifier flags it, a human reviews. The logic is sound. The failure is runtime.

Chat has a 50ms latency budget. Feed has 200ms. Those numbers aren't arbitrary – they're the point at which a slow response stops feeling like a response and starts feeling broken. If the safety classifier runs inside that window, you get synchronous review. If it misses, you have two options: show the content anyway, or hold it for async review. Either way, you've changed your policy. Not in a document. At runtime.

I built a small demo to make this tradeoff concrete. Drag the slider left and the classifier finishes at 30ms, comfortably inside the chat window. Flagged content goes to synchronous review before anything surfaces. Drag it right and latency climbs past 50ms. The classifier misses its window, and async review kicks in. Content that should have been held gets shown, or held content creates a noticeable delay. The policy you thought you had – the one that says harmful content never surfaces without review – isn't violated by a bad actor. It's violated by a slow P99.

This matters because latency rarely gets treated as a policy variable. It gets treated as an infrastructure problem. Engineers optimize it, but the safety team didn't sign off on what happens when optimization fails. Nobody wrote down that a classifier running at 80ms during a traffic spike means async review, and nobody told the policy team that async review means some percentage of flagged content goes live before a human sees it. The gap between the policy as written and the policy as executed is a runtime property.

Working on cross-org AI model integration has made this gap harder to ignore. When model dependencies cross team boundaries, the latency budget gets sliced by each hop. A classifier that runs at 40ms in isolation runs at 70ms when it's waiting on an upstream embedding service. The trust assumptions from the design review don't survive contact with production topology.