under-the-hood
6 articles tagged “under-the-hood”.
LLM Cost Reduction: How We Cut Our Inference Cost per Session by 6x
We build cost attribution for AI agent teams, then turned it on ourselves. How we cut LLM cost per session roughly 6x in five weeks, and what wasn't efficiency.
No Way to Merge
Our incremental clustering pipeline kept creating duplicate issues and had no way to merge them. Why tuning the matcher wasn't enough, and the merge job we added.
We stopped paying an LLM to answer yes/no questions: 21x cheaper with Jev
We moved session classification off a generative LLM and onto Jev. Cost went from $89 an hour to $4. Here's where the money was going and where we're cutting next.
Stop Asking the Model What the Categories Are
We rebuilt issue clustering by moving the taxonomy out of clustering entirely — into detection. What broke in the first version, what the rewrite bought, and what it cost.
ClickHouse Text Indexes, Direct Read, and the Compatibility Trap
We shipped a text index that did byte-for-byte nothing while every diagnostic said it was working. The culprit: one compatibility setting silently pinning a prerequisite off.
The Granularity Problem, or: Why Your Embeddings Don't Know What a "Third-Party Integration" Is
Two VPs want the same dashboard at different zoom levels. Why dendrogram cuts can't give it to them — and what to build instead: leaf dedup, extracted facets, and a declared taxonomy.