Claude Code Analytics: Why Anthropic Underreports AI Impact

"How much of this pull request did the AI actually write?"

"The dashboard says 80%."

"But you spent three hours refactoring the logic and renaming the variables."

"Yeah. I basically rewrote the whole thing."

This conversation is playing out in engineering teams everywhere. Most AI coding tools try to claim as much credit as possible to justify their enterprise seat cost. If an AI agent touches a file, the analytics dashboard claims a win. But when toolmakers exaggerate their impact, developers push back. They know when they did the heavy lifting. Seeing a dashboard claim 90% AI-generated code for a file they spent hours debugging feels like credit theft.

Anthropic is taking a completely different approach. With the release of their new analytics capabilities for Claude Code, they are doing something counter-intuitive. They are deliberately underreporting their AI impact to protect developer trust.

The Vanity Metric Trap

For an AI Product Manager, the pressure to demonstrate return on investment is immense. When an enterprise customer pays thousands of dollars for developer seat licenses, they expect a clear, quantifiable return. This pressure creates a dangerous incentive structure. Product teams are pushed to design dashboards that make the AI look as productive as possible. If the AI suggests a block of code, and some version of that block eventually gets merged, the system claims full credit.

This is the vanity metric trap. In traditional software, counting clicks or pageviews worked because a click represented a clear, binary user action. In the probabilistic world of AI, these metrics fall flat. We have written before about how traditional SaaS metrics fail for AI agent analytics. When you measure AI impact using the same blunt instruments, you end up with dashboards that look fantastic on paper but bear no relation to reality.

To fix this, we have to look past simple uptime or surface-level actions. Building a reliable system requires true AI agent observability, which means understanding why a model made a specific decision rather than just tracking how many times it ran. Without this depth, traditional AI coding metrics (like lines of code accepted or raw PR counts) paint a dangerously misleading picture. These legacy metrics get several things wrong:

• They measure volume rather than utility. A developer might accept 500 lines of boilerplate but spend two hours rewriting the 10 lines of core logic.
• They ignore cognitive load. Reviewing and refactoring AI-generated code often takes more mental effort than writing it from scratch.
• They reward noise. An AI that generates verbose, repetitive code looks more productive on a lines-of-code dashboard than one that writes a clean, elegant helper.

The danger of these inflated metrics is that they hide the actual friction in the developer workflow. If your dashboard tells you that your team is writing code 80% faster, but your release velocity is flat and your bug count is climbing, the metrics are lying to you. This is similar to why optimizing for 100% task completion is dangerous in agent evaluations. An agent might technically complete a task by writing code that compiles, but if that code is poorly structured or introduces subtle regressions, the human developer has to spend hours rewriting it. If the telemetry still counts that as a complete win for the AI, the product team loses visibility into the actual developer experience.

How Anthropic Measures Claude Code Usage

Anthropic's new dashboard rejects this vanity model. Available via the Anthropic Console, the Claude Code analytics dashboard provides leaders with visibility into team-wide adoption, showing how teams interact with the tool without inflating the numbers. Instead of trying to maximize value-delivered metrics, the system is designed to protect developer trust.

When engineering teams evaluate claude code usage, they want to know if the tool is actually helping, or if developers are just spending their days babysitting an AI. If the tool claims credit for work a human heavily edited, the human will reject the tool.

To prevent this, Anthropic's dashboard is built on a foundation of conservative attribution. They would rather underreport Claude's impact than risk a single false positive that alienates a developer. This philosophy shifts the focus from marketing fluff to genuine utility.

The 20% Rewrite Threshold

The mechanics of this conservative attribution are fascinating. As detailed in Larridin's analysis of the Claude Team Plan, the new dashboard deliberately excludes any code that developers rewrite by more than 20% from its PR attribution metrics.

How does this 20% threshold work in practice? Let's break down the math:

1. Code generation: Claude writes a 100-line function for a new API endpoint.
2. Developer modification: The developer reviews the code, finds a couple of edge cases, and rewrites 25 lines to handle custom error types.
3. Attribution calculation: Because the developer rewrote 25% of the code (which is over the 20% threshold), the system discards the entire block from Claude's contribution metrics.
4. Final report: The dashboard attributes zero lines of that function to Claude.

This 20% rewrite threshold is an incredibly high bar for an AI tool to clear. In a typical development workflow, even a senior human developer rarely writes code that remains completely untouched by peers during a code review. We expect refactoring. We expect minor tweaks to variable names, logging statements, or import blocks. By setting the threshold at 20%, Anthropic is acknowledging that if a human has to make significant changes to the generated code, the AI did not actually complete the task.

This approach is a massive departure from typical AI coding metrics. Most tools use simple diff matching. If a line of code in the final pull request matches a line generated by the AI, the tool claims credit. But diffs are noisy. A developer might keep a few boilerplate lines while completely rewriting the core logic. Under typical metrics, the AI gets credit for the whole block. Under Anthropic's new rules, that block is disqualified.

The Strict Time Window

The conservative design does not stop at the rewrite threshold. Anthropic also applies a remarkably strict time window to its attribution matching to measure how claude code usage translates to actual merged code.

According to the same documentation, matching only considers sessions from 21 days before to 2 days after the merge. The strict time window operates under similar constraints:

1. Session window: The dashboard only matches Claude sessions that occurred within 21 days before the merge.
2. Post-merge window: It also monitors changes up to 2 days after the merge to catch quick hotfixes.
3. Discarding long-tail noise: Any session older than 3 weeks is ignored, even if some of the code looks similar. This prevents the tool from claiming credit for accidental similarities in boilerplate code.

This temporal precision is especially important because of how developers actually use AI tools. A developer might run a Claude Code session to explore a new API or test an idea, but then put that work aside for several weeks. When they finally return to build the feature, they might write the code from scratch, using only a few concepts they learned during their initial session. In a loose telemetry system, that later PR would be flagged as AI-influenced because of the historical session. Anthropic's 21-day limit ensures that only active, immediate contributions are measured.

Why Conservative AI Coding Metrics Build Trust

For AI product managers and engineering leaders, this is a masterclass in product psychology.

Building tools for technical users is different from building for the general public. Developers are naturally skeptical of marketing claims. They live in a world of precise inputs and outputs. If you show them a dashboard that says an AI tool is doing 80% of their work, when they know they are still working 40-hour weeks to fix AI-generated bugs, they will lose faith in the tool and the team pushing it.

Why does this conservative approach work so well for technical teams?

1. It validates human effort: When developers look at the dashboard and see their own contributions valued, they do not feel like a machine is erasing their work.
2. It makes the metrics undeniable: When the dashboard claims that Claude wrote 30% of a repository, the team knows that 30% was clean, high-quality code that required almost no human intervention.
3. It exposes real workflow friction: By removing the padded numbers, engineering leaders can see exactly where the tool is struggling. If attribution is low on a repository, it highlights a need for better context or cleaner interfaces.

This is why we focus so heavily on high-fidelity telemetry at Brizz. Whether you are building internal developer tools or customer-facing agents, your analytics must reflect reality, not a marketing team's wishlist. If you are interested in how we help teams build this kind of deep, trustworthy visibility into their AI systems, you can read more about our approach to developer tool telemetry.

Summary: High-Fidelity Analytics for AI Agents

To help engineering leaders and AI product managers navigate these new telemetry standards, we compiled answers to the most common questions about how these metrics are calculated.

Frequently Asked Questions

How does Claude Code attribute PRs? Claude Code attributes pull requests by matching lines of code merged into your main branch against active sessions inside the Claude Code CLI. To ensure accuracy, the system only counts code that remains largely intact and is merged within a strict temporal window.

What is the Claude Code rewrite threshold? The rewrite threshold is set at 20%. If a human developer modifies more than 20% of the code suggested by Claude in a given block, the entire block is discarded from Claude's contribution metrics. The dashboard will attribute zero lines of that block to the AI.

How does the 21-day window affect Claude Code analytics? The dashboard only attributes code to Claude if the corresponding CLI session occurred within 21 days before the merge. It also tracks the repository for 2 days after the merge to verify that a developer did not immediately rewrite or hotfix the generated code. Any session older than 21 days is excluded from attribution to eliminate long-tail noise and accidental matches.

The Bottom Line

Protecting developer trust requires metrics that measure genuine value, not vanity touchpoints. Anthropic's conservative attribution model for Claude Code shows that underreporting AI impact is a deliberate strategy to build long-term credibility, rather than a failure of marketing. When you build AI tools for technical users, the fastest way to get rejected is to take credit for human work.