Test your core by building the thing that uses it, then throwing it away

The cleanest signal that a library or API is good isn't a passing test suite—it's whether someone else can build something useful on top of it without reading your source code. Ephemeral testing, as Daniel Lemire describes it, makes that signal systematic: have an AI agent build a layer on top of your component, evaluate what the agent produced, then throw the whole thing away.

The test artifact is discarded. The verdict stays.

Why this works

Unit tests are written by the same person who wrote the code. That means they tend to encode the author's mental model, including blind spots. A library with hidden state or surprising defaults will pass its own unit tests just fine—the author designed both.

Building on top of the library, even ephemerally, forces a different perspective. As Lemire puts it: a clean API with stable invariants lets an agent produce something that works quickly; a library with incomplete docs or unexpected behavior produces a pile of patches and failures. Those failures are evidence about your code, not about the agent.

This is also a good way to stress-test API design decisions before you commit to them. If an agent can't build a plausible consumer without significant friction, a human developer won't fare much better.

The trade-off is real

Ephemeral testing isn't a replacement for deterministic regression tests. A unit test that passes today will pass tomorrow under the same conditions. An ephemeral test driven by an AI agent introduces variability—different agents, different runs, different results. Lemire notes you can repeat the process with different agents and tasks, which helps, but it doesn't make the signal crisp in the way a binary pass/fail test does.

The technique also requires that building the ephemeral layer actually be cheap. That's where AI changes the calculus. Before agents could scaffold a working prototype in minutes, throwing away a full integration layer after each test cycle was impractical. Now it isn't, and that's the real enabling condition here.

How we'd use it

We see the most value in two places. First, at the point of API design—before the interface is frozen, build a quick throwaway consumer and see what breaks. Second, when evaluating a significant refactor—rather than asserting that your internal behavior is unchanged, assert that the experience of building on top of it is unchanged.

What we wouldn't do is treat it as a substitute for a regression suite on a production system. The ephemeral layer tells you about ergonomics and surface design. It doesn't tell you whether a specific edge case in your internals still behaves correctly after a change.

The name matters too. Calling it

Sources

• Ephemeral testing (original)
• Found via the TLDR newsletter