Blog
Insights on AI agent analytics, product management for AI, and building better AI products.
AI Agent Sprint Planning: Prioritize with Capability Success
AI agent sprint planning is broken by raw log noise. Learn how tracking capability success rates helps AI PMs prioritize fixes and drive roadmap impact.
Every Analytics Tool Now Reads Agent Conversations. Here's What Comes Next.
Amplitude, Mixpanel and PostHog now all read agent conversations. The harder part is what happens next: shipping the fix and proving it worked.
Solving the Manage Claude Code Agents Problem: Swarm Economics at Scale
Anthropic's 1,000-agent dynamic workflows solve code accuracy but introduce massive token burn. Here is how to manage swarm economics, concurrency, and multi-agent observability at scale.
How We Made Long-Running AI Jobs Survive Worker Shutdowns
Long-running AI clustering jobs kept dying when Cloud Run scaled workers down. Splitting them into small idempotent slices turned shutdowns into cheap retries.
AI Evaluation Frameworks: Why State-Based Testing Wins
Learn why AI builders are moving beyond text outputs and tool logs to test agents on backend state and repeated runs, and what a new Microsoft benchmark found.
The Future of AI Product Management: Building Self-Improving Products
Master the future of AI product management. Learn the 5-part AI agent architecture behind self-improving products and how the PM role is shifting.
AI Agent Architecture: Why Bridgewater Treats Agents as Compilers
Treating AI agents as compilers rather than chatbots improves speed and reliability. Learn how Bridgewater uses parallel execution and DAG validation to hit 90% accuracy.
LLM Cost Reduction: How We Cut Our Inference Cost per Session by 6x
We build cost attribution for AI agent teams, then turned it on ourselves. How we cut LLM cost per session roughly 6x in five weeks, and what wasn't efficiency.
No Way to Merge
Our incremental clustering pipeline kept creating duplicate issues and had no way to merge them. Why tuning the matcher wasn't enough, and the merge job we added.
AI Agent Failure Handling: Solving Multi-Agent Latency
Effective AI agent failure handling requires moving past manual traces. Use deterministic indicators to identify AI performance bottlenecks and latency.
We stopped paying an LLM to answer yes/no questions: 21x cheaper with Jev
We moved session classification off a generative LLM and onto Jev. Cost went from $89 an hour to $4. Here's where the money was going and where we're cutting next.
Claude Code Analytics: Why Anthropic Underreports AI Impact
Anthropic's new Claude Code analytics dashboard uses conservative attribution to protect developer trust. By excluding code rewritten by more than 20% and limiting matching to a strict 21-day window, they show how high-fidelity telemetry beats vanity metrics.
Braintrust Pricing vs. the Alternatives: What Full Trace Coverage Actually Costs
Five observability platforms, five incompatible billing units. What Braintrust pricing really costs at production volume, how the alternatives compare, and how to model the number for your own traffic.
Stop Asking the Model What the Categories Are
We rebuilt issue clustering by moving the taxonomy out of clustering entirely — into detection. What broke in the first version, what the rewrite bought, and what it cost.
The Multiplicative Failure Trap: Why Multi-Tool AI Agents Crash Midway
Why multi-tool AI agents scraping websites, inboxes, and SaaS tools keep failing midway, and how state checkpointing stabilizes multi-step workflows.
ClickHouse Text Indexes, Direct Read, and the Compatibility Trap
We shipped a text index that did byte-for-byte nothing while every diagnostic said it was working. The culprit: one compatibility setting silently pinning a prerequisite off.
The Granularity Problem, or: Why Your Embeddings Don't Know What a "Third-Party Integration" Is
Two VPs want the same dashboard at different zoom levels. Why dendrogram cuts can't give it to them — and what to build instead: leaf dedup, extracted facets, and a declared taxonomy.
Detect LLM Application Blind Spots Scorers Miss
Automated LLM scorers create a false sense of security through high raw agreement rates. Learn how to find evaluation blind spots using Cohen’s Kappa and adversarial testing.
Mastering Parallel Claude Code Sessions: A Guide to Multi-Agent Workflows
Six terminals open and no idea which agent is waiting on you. How to run parallel Claude Code sessions with fleet.
Agent Evaluations: Why Output-Only Checks Fail
Evaluating AI agents on final output hides silent failures and loops. Learn why trajectory evaluation and observability are essential for reliable agentic workflows.
Agentic AI Observability: Why Sampling Fails AI Agents
Traditional APM sampling fails for probabilistic AI. Learn why agentic AI observability requires 100% visibility and wide events to fix silent failures.
Why 100% Task Completion is the Most Dangerous Metric in Agent Evaluations
Task completion is a dangerously misleading metric for AI agents. When agents are optimized to succeed, they will sometimes secretly alter the environment or rewrite test suites to guarantee a pass. Here is how to build a sabotage-proof evaluation framework.
Conversational AI vs Search Bar: Building the Future of Retail
Amazon is replacing search bars with conversational AI. Learn what this format shift means for AI builders and how to measure conversational agent success.
AI Agent Error Handling: Why Strict Tool Standards Matter
Fix silent failures with a strict AI agent error handling contract. Learn how structured tool errors improve debugging and observability for LLM agents.
RAG Retrieval Optimization: The #1 Mistake AI Builders Make
Upgrading your LLM to fix hallucinations is a costly trap. Learn how RAG retrieval optimization and key evaluation metrics solve the real bottleneck.
AI Product Management: Fixing Silent Model Degradation
Stop AI model drift. Learn why AI product management requires continuous evaluation of probabilistic systems to prevent silent failure and user churn.
AI Agent Analytics: Why Traditional SaaS Metrics Fail
Stop slicing AI agent data by industry or ARR. Learn how intent-based AI agent analytics surface real product gaps and improve agentic workflow performance.
LLM-as-a-Judge Cost: Surviving the Production Compute Tax
Scaling AI agents? LLM-as-a-judge costs can destroy your unit economics. Learn 4 strategies to optimize token usage and reduce LLM evaluation cloud spend.
Continuous Evaluation for AI: Why Traditional QA Fails
Stop silent AI degradation. Learn why continuous evaluation for AI is replacing traditional QA to monitor semantic drift and maintain LLM product quality.
LLM as a Judge: Why Eval Scores Fail Your AI Roadmap
LLM as a judge is great for regression testing but terrible for discovery. Learn why high eval scores hide flat retention and how to surface real user intent.
Introducing the Brizz MCP Server: Your AI Analytics, One Question Away
Ask Brizz anything about your AI product — directly from Claude, Cursor, or any MCP client. No dashboards, no digging, just answers.
Mission Control for Claude Code: How to Manage 10 Agents Without Losing Your Mind
Stop tab-switching and start orchestrating. fleet is a terminal mission control for managing parallel Claude Code sessions.
Introducing Brizz: Product Analytics for AI Agents
Traditional analytics stop at clicks and pageviews. Learn how Brizz helps teams understand AI agent behavior, improve quality, and make data-driven product decisions.