Dishonest by design: the verification gap in AI products

The argument is simple and damning: AI products disclaim their unreliability in fine print while designing every interaction to discourage the verification that disclaimer requires. That's not a product oversight. It's a design choice.

Glyph's essay lists specific features that would signal a product takes its own limitations seriously: error-checking workflows built into the interface, citations presented as primary content rather than footnotes, no first-person language, task-specific UI rather than open-ended chat, visible context management, and sandboxed agentic operations. None of these are technically hard. All of them would slow down the demo.

That's the trade-off worth naming. A product that makes verification easy will feel slower and more effortful than one that doesn't. Users notice friction before they notice value. For small teams shipping AI features, the pressure to hide that friction is real.

But hiding it means shipping misleading software. The disclaimers current products display — "AI can make mistakes, please check responses" — are designed to transfer legal liability, not help users actually check. If your product tells someone they need to verify output but gives them no workflow for doing it, you've handed them a problem and taken away the tools.

The citation argument stands out most. Glyph's point is that when AI is used for research, citations should be the primary content — displayed large, with full metadata and literal unmodified quotations extracted by regular code, not a language model. AI summaries should appear small and secondary, de-emphasized until the user confirms accuracy. This inverts how every current research tool is built. It also inverts what makes those tools impressive in a demo: the confident synthesized answer.

For tools focused on professional workflows — which is where we work — this inversion is correct. A professional using a research tool is responsible for the accuracy of what they produce. Making them feel confident when they shouldn't isn't a feature; it's a liability transfer.

The agentic coding critique is equally direct. "Best practice" advice to contain coding agents with Docker containers or MCP approval gateways is inadequate by design. It turns the user into an approval automaton until they hit "yes to all." A serious product would sandbox filesystem operations independent of the prompt, enforce repo snapshotting on every operation, and remove auto-approve modes. These are engineering decisions, not unsolved problems.

One place we'd push back slightly: the call for non-natural-language interfaces for specific tasks is right in principle but expensive in practice for small teams. Task-specific UI for every AI capability takes real build time. The practical move is probably to start with structured UI for your highest-stakes outputs — where a wrong answer creates real cost — and treat open chat as a scratchpad, not a source.

The essay's closing argument is pointed: adding these features would make the tools less impressive by showing users how often the model is wrong and how much time they're actually spending. Glyph believes the frontier labs know this and have made a deliberate choice not to build them.

We don't need to resolve that question to know what to do with our own products. Build the verification layer. Make it visible. Treat honesty as a feature, not a disclaimer.

Sources

• What Would A Serious AI Product Look Like? (original)
• Found via the TLDR newsletter