How much of what you shipped last quarter came from a production line?
Here's a question I've started asking engineering leaders, and it makes almost everyone uncomfortable:
Of the features your team shipped last quarter — what fraction came off a production line, and what fraction was handcraft?
By production line I mean: a checkable spec existed before code, verification was automated, and the work ran in parallel tracks that one person could supervise. By handcraft I mean: an engineer typed it line by line and a human eyeball was the test suite.
Most leaders can't answer. Not approximately, not at all. They can tell me which AI tools they bought and roughly what those cost. They can tell me things "feel faster." But the ratio — the actual structure of their delivery capacity — is invisible to them.
That used to be fine. It isn't anymore.
The factory era arrived without asking
One data point crystallized this for me. In February 2026, Anthropic's engineering team published how 16 parallel Claude agents built a working C compiler from scratch: two weeks, ~2,000 sessions, about $20k in compute — 100,000 lines of Rust that compiles the Linux kernel and passes 99% of the GCC torture test suite. That's not a demo; that's a production line. The direction is unambiguous — code generation has become cheap and parallel, and the teams that industrialize it will rebuild feature parity against the teams that don't, at a fraction of the cost.
Here's the part most commentary misses: the factory itself is not a moat. Playbooks circulate, models are rented, a production line can be rebuilt in months. What differs between teams — what compounds — is the quality of the system around generation: specs, context, verification, review gates. The compiler story works because a perfect verification oracle already existed (GCC's test suite). Your business does not have a ready-made oracle. Building one is the actual work.
Which means the first question isn't "which AI tools should we buy?" It's: what is our factory, today, measured honestly?
A maturity scale you can locate yourself on
We built a measurement framework for this — four metrics, sampled from your own repos and CI, no vendor access required. It grades a team 0 to 4. Here's the free preview — find yourself in this table:
| Level | Name | What it looks like day-to-day |
|---|---|---|
| 0 | Pure handcraft | Requirements travel by mouth; testing is a human clicking; one person, one task at a time |
| 1 | Tool-assisted handcraft | AI is a faster typewriter — autocomplete and chat help people type, but decisions and acceptance flow exactly as before |
| 2 | Semi-production-line | "We have a standard way to do this kind of feature" — but the standard lives in heads and varies by person |
| 3 | Mostly production-line | Spec, generation, and verification clearly separated; most features run one repeatable flow; humans mostly review |
| 4 | Near-fully automated line | Humans act as partners and verifiers, not typists; only genuinely new feature types need new process design |
Two honest observations from building this:
Most teams that adopted AI tools are at Level 1 and believe they're at Level 3. The tools made typing faster; nothing downstream — how work is specified, verified, accepted — changed. That gap between felt progress and structural progress is exactly why "we can't quantify the AI productivity gain" keeps coming up in board meetings.
The teams that will be hard to compete with aren't the ones with the most AI spend. They're the ones where verification is automated enough that adding parallel generation actually compounds instead of adding review burden.
Measure it yourself
We turned the framework into a self-assessment kit: a scorecard with exact sampling instructions (which PRs to pull, what counts as a spec, the self-deception traps for each metric), a guide that tells you what your level means and the one next step for it, and a 90-day action-plan template you can re-measure yourself against.
It takes one engineering lead 30–60 minutes with your own data. Nothing connects to your systems; your data never leaves your machines.
→ Get the self-assessment kit here: https://useassay.gumroad.com/l/ymolaf
That's the only thing I'm selling in this post. If you run it and your level surprises you — in either direction — I'd genuinely like to hear about it: hello@useassay.co.
Who made this, and why the name is "Assay"
Full transparency, because it's the point.
Assay is the metallurgical test that tells you an ore's purity and value. You cannot tell gold from pyrite by looking at it — you take a sample, put it through fire, and weigh what survives. That's the name because that's the business: the market is filling up with AI agent libraries, collections of prompts and personas that nobody has ever tested. Collecting is cheap. Assaying is the work.
So: the methodology in this kit was distilled and stress-tested by Assay's AI team — agents operating under a written constitution, whose work is graded by a separate examiner agent that didn't write it, before anything ships. That examiner is ours, not a third party, and I'll say so plainly rather than dress it up as certification. It is, however, allowed to fail us: this kit's methodology failed its first exam at 72/100, was rewritten, and passed at 98/100. We keep both reports, and I'll send you either one if you ask.
I reviewed and signed off on it as the accountable human. Assay is a brand, not a corporation — no office, no staff roster; a method, a record, and one person who answers the email. Showing you our quality-control record beats telling you we're credible — and if the idea of an AI team with an audit trail intrigues you, that's not an accident either.