The longer I write code with AI, the more one question refuses to go away: how do you know a person is still thinking. The code came from AI, the plan came from AI, the review was cleared by AI, everyone looks productive, and yet where a person's own judgment went is something fewer and fewer people can explain. And that pile of AI metrics we are holding never intended to answer the question in the first place.
A premise nobody ever stated
Almost all of the AI metrics we chased over the past year were designed on a "more is better" basis, encouraging everyone to hand as much of their work to AI as possible: the share of code generated by AI, for example, the higher the better. Taken one at a time these goals sound fine, but they all share a premise nobody ever stated out loud: the person using AI already has judgment.
That premise often does not hold. The clearest case is a new hire. Judgment has not been built yet, and you tell him to maximize contribution rate, maximize tokens, maximize generated share, which is pushing him toward an endpoint where the human only verifies. But verification is itself a form of judgment, and judgment only grows from doing the work yourself. Ask someone who cannot yet judge to verify, and the only thing he can actually do is approve. He thinks he is gatekeeping. He is waving things through with his eyes closed.
And this is not only about seniority. An engineer with ten years behind him, dropped into a codebase he has never touched and a business domain he does not know, starts from zero all the same. He is not short on the ability to write code. He is short on this environment's historical decisions, technical debt, business logic, and the unspoken working style of the teams he depends on. None of that is in the docs. It only grows from soaking in it and doing the work. And verification runs on exactly that. So as long as a person still needs onboarding, however senior he was before, he should not skip straight to verification.
Here is where it gets interesting: swap the premise from "the person already has judgment" to "the person is building judgment," and that whole pile of "more is better" metrics has to be read backwards.
What reading it backwards means
The code metric we actually hold onto is one: AI code contribution rate, the share of code merged into master that originated from AI. Reading it backwards is simple. It is the share of hand-written code, and it corresponds to the "flying without the network" exercise: can this person still write code once AI is taken away.
That metric used to be read differently, though. We would break it down one level further and look at how the AI code was produced, roughly two categories: completion, the old Copilot-style tab completion, and generation, using an agent to write code directly. The two represent two behavioral modes, and we thought generation was better than completion, because it meant AI's contribution went deeper and the human had already retreated to verification.
Read backwards, the split still has value, only good and bad trade places. A high completion share means the human is leading: intent, structure, and decisions are yours, AI only fills in tokens, and that cognitive participation is exactly where judgment comes from. A high generation share means the human has retreated to the verifier's seat, and the comprehension debt was taken on at the input side. So the old line, "higher contribution rate is better, higher generation share is better," reads backwards as more hand-written is better, more completion is better. Three points make one spectrum: hand-written, completion, generation. Which cell someone falls into tells you roughly what stage he is in and whether he is using AI correctly.
The completion-versus-generation split is the sharpest of the lot, because it measures who is authoring, and authoring is exactly where judgment is built. Better still, this split does not require going back through git to infer who wrote which line. Completion and generation are recorded at the moment of creation: Copilot's usage metrics treat code_completion and agent_edit as two separate feature categories, and Cursor's Analytics API simply splits /tabs (completion) and /agent-edits (generation) into two endpoints, both exportable per person. This ruler is not something I imagined. It is already a product feature.
Read backwards, it becomes a gate
Once you can read this progress chart, the next step is to turn it into a gate: a new hire's token consumption should not be large, his contribution rate should not be high, and his generation share should be forcibly held down. Open it up once onboarding is over, or once a leader signs off. Using little AI is not a bad thing early on. It is building cognition and judgment the honest way.
There is a property here I keep turning over: this gate is very hard to cheat. If a new hire wants to push his completion share up, the only way is to refuse generation and write the code himself, which is exactly what the gate wanted to force him into. The way to game this metric is to complete the training it was trying to force. Most metrics stop working the moment they get gamed. This one is the opposite: getting gamed means getting satisfied. That is also why "completion versus generation" is worth using as a gate rather than as a number you glance at.
The gate does have one practical awkwardness. Completion versus generation is a beautiful metric, but people are now used to generating code directly with agents like Claude Code and Codex, and fewer and fewer people still use tab completion in Copilot or Cursor. Actually using it as a gate would mean forcing everyone back into completion-style editors, and that in turn cannot reuse the existing harness infrastructure. So when it comes to rolling this out, we may still need to find some other proxy metric.
Is high token consumption good or bad
Every team seems to have that one person: enormous token consumption, one person doing the work of ten. I have always been curious how that is done. At the same time I have always had a vague sense that more token consumption is not better. It took me a long time to realize those two things are the same thing.
First, why I doubt "more is better." When a person works with AI, the cognitive investment sits at the two ends: upstream you have to work out what you want and what the standard is, downstream you have to confirm whether what came out is right and then digest the result into your own understanding. The execution in the middle can be outsourced to AI. Since both ends still rest on the human, and human attention and judgment have a physiological ceiling, token consumption should in theory have a ceiling too. There is a limit to how many drafts an editor can truly read in a day, and a limit to how many deals an investor can truly diligence in a year. When someone really does consume ten colleagues' worth of tokens, my first reaction is that he cut out one of the ends: firing off prompts before thinking upstream, or taking the output straight without verifying downstream.
I call that state surrender mode. It is the opposite of augmentation mode. In augmentation mode AI amplifies what you were already doing. In surrender mode you expect AI to produce value out of nothing, and you outsource the share of cognitive labor that was supposed to be yours. So token consumption is less a capability metric than a cognitive-investment metric: too much makes me wary, and too little is not necessarily good either, since it may mean AI is not being used at all, or that someone is still accumulating the honest way.
But the same number can tell the opposite story. Someone whose token consumption far exceeds his colleagues' may be there precisely because he is a level above: he has collapsed his judgment into something reusable, a spec written through, a skill worked out and saved, a workflow polished over many passes, a loop that runs on its own. The judgment did not disappear. It moved out of the improvised thinking scattered across every task and into the standard, so each unit of judgment now levers many times the execution. Token counts that high rest on amplified leverage, and he may genuinely be doing the work of ten.
The industry has already had this fight. Meta at one point had an internal leaderboard ranking employees by token consumption, reportedly covering eighty-five thousand people, with the top spot burning 281 billion tokens in a month. Once it got out it drew a wave of criticism, and people gave it a name: tokenmaxxing, the token version of lines of code, measuring activity rather than value. Jellyfish ran the numbers on more than seven thousand engineers: the top 20% by token consumption produced twice the PRs of the bottom 20%, at ten times the token cost. Faros AI tracked twenty-two thousand developers: task volume up 34%, bugs up 54% alongside it, review time five times higher. The industry's conclusion converges on roughly one line: treat token consumption as a "more is better" KPI and it will get gamed.
Coming all the way around, I have to admit this is a question I cannot answer. Surrender and being a level above, one cutting judgment out and the other amplifying it, look identical in the token number. You cannot tell them apart from the number alone. So we should stop agonizing over this metric: it does reflect something, but it yields no conclusion. It is a signpost, not a verdict.
Then stop looking at output and look at your input
Having gone around token consumption, I want a more fundamental question: where did this person's cognition actually go.
The AI code contribution rate from earlier already hides half the answer. The more AI contributes, the less the human contributes. Read backwards, wherever AI contributed less, the human contributed more and the cognitive investment was heavier. So the most direct idea is this: break a piece of work apart, measure the AI contribution rate of each piece, then read it backwards. Wherever the contribution rate is low is where this person was actually thinking.
But the idea gets stuck on a reality: code is a very special kind of output, relatively standardized with clear boundaries, which is why an AI contribution rate is measurable at all. It is hard to compute which parts of a document were written by AI and which by a human. In meetings and conversations with people, even the "output" is scattered across dialogue with no place to put a ruler. Inferring cognition from AI contribution rate holds only for this one artifact, code, and stops holding for other kinds of work.
So I stepped back: stop measuring AI's output, measure the human's output instead, since a human's output is the direct expression of cognitive investment. That path does not work either: a human's output is not 100% capturable, and that one hour you spent talking with someone is output that leaves no trace.
Step back once more, onto something you can actually grab: the part of a human's output that was fed to AI as input. In plain terms, count the words you actively said to the agent. Note the line to draw here: what gets counted is not the prompt, because a prompt may have been injected with a mass of passively stuffed content, a whole document, a full error log. What gets counted is the part you wrote yourself, and that is the written record your cognition left behind.
What makes this metric interesting is that it steps around every dead end above. It is something a human wrote, so it has a physiological ceiling and cannot inflate without limit the way AI output does, which means it can actually be reviewed. Maybe some day a "prompt review" will appear, just like today's code review. It is also closer to cognition than a contribution rate: whatever you actively write in front of the agent is roughly where your attention landed.
Things I have not figured out yet
What "human output, AI input" should be called and which metrics should measure it, I have not worked out. The easiest thing to grab is volume: how many words a person wrote, how many tokens got fed in. But volume runs into the same wall as token consumption above: large volume may be a carefully prepared set of constraints, or it may be a giant blob of context dumped in for AI to figure out on its own, and the number alone cannot tell them apart. Frequency is the same: high frequency may be repeated verification, or it may be the same underspecified request retried dozens of times. What actually discriminates is what these inputs are doing, setting constraints, verifying, exploring, or executing, but that requires semantic classification of content, which telemetry does not hand you directly. Maybe there is something usable in here. I will come back to it when I have an idea.
And a counterfactual: if a team really ran on this gate for a year, would the judgment of the people newly arrived in a new environment (not necessarily juniors) actually grow faster than in a control group with AI unrestricted? This is the part of the whole framework that most deserves validation and is hardest to validate, because independent review performance after onboarding ends is itself not easy to quantify.
References
- GitHub (2026). Copilot usage metrics. Official documentation. The feature dimension of usage metrics distinguishes token categories such as code_completion and agent_edit; the code generation dashboard reports "Agent contribution" (the agent's share of lines added and removed).
- Cursor (2026). Analytics API. Official documentation. Two endpoints, /analytics/team/tabs (completion) and /analytics/team/agent-edits (generation), filterable by user, both enterprise-tier.
- Faros AI (2026). Tokenmaxxing: Why AI token consumption isn't engineering productivity. Blog. Tracking 22,000 developers: task volume +34% but bugs +54% and review time 5x; treating token consumption as a productivity metric is a remake of the lines-of-code mistake.
- Jellyfish (2026). AI Coding Productivity Study. A Q1 2026 study of 7,548 engineers: the top 20% by token consumption produced 2x the PRs of the bottom 20%, at 10x the token cost.
- Meta (2026). Internal token consumption leaderboard (reported in 2026 by outlets including LeadDev and PeopleManagingPeople). Ranked roughly 85,000 employees by token consumption, with the top consumer at 281 billion tokens in a single month, criticized as a vanity metric.