No. 01 — Research synthesizer
Turns hours of user interviews into ranked themes, where every insight links to a real quote, and made-up quotes are thrown out.
The problem
Interview synthesis takes days, and AI summaries make up quotes, which teams then base product decisions on.
What I built
A pipeline that extracts, verifies, clusters and ranks insights. Each one traces back to the exact quote and, for recordings, the moment it was said.
The result
Tested on 125 real transcripts. Unverifiable quotes are dropped and the drop rate is published.
01See it work
- 125
- real transcripts parsed in testing
- 8
- pipeline stages, from parsing to Q&A
- 24
- backend tests on real Postgres + pgvector
- 0
- API keys stored server-side, by test
02Why it matters
After a round of user interviews, someone spends days tagging quotes and grouping them into themes. AI can do it in minutes, but it paraphrases and sometimes makes up quotes. In research that's fatal, because product decisions get made on those quotes.
03The idea
Traceability is the product. Every quote the model extracts is checked against what that participant actually said. If it can't be found, it's dropped, and the drop rate is shown, so you can see how much to trust the output.
04How it works
- DeterministicParse.txt / .vtt / .srt, or audio via faster-whisper
- AI modelExtractClaude Haiku with a forced tool call returns pains, needs and quotes
- Verification gateVerify quotesFuzzy match inside the participant's own turn; drop if not found
- DeterministicClusterbge-small embeddings, HDBSCAN per insight type
- AI modelName & rankClaude Sonnet names themes, ranked by reach × severity
- OutputReport & ask450-word report, charts, and Q&A with hybrid retrieval (RRF)
05Decisions that mattered
Drop, don't trust
A paraphrased quote looks fine and is wrong. Matching against the speaker's own turn catches misattributed lines, not just made-up ones.
The right model for each step
Haiku handles high-volume extraction cheaply, and Sonnet does the judgment calls like naming themes. Without an API key, a heuristic baseline still runs end to end.
Clusters that can say "noise"
HDBSCAN doesn't need a preset number of themes, and it leaves one-off comments unclustered instead of forcing them into a theme.
Your key stays yours
Bring-your-own-key lives in the browser, and the server holds it in memory only. A test proves no database table ever stores it, and error messages are scrubbed of it.
06How I know it works
The plan: run Claude and the heuristic baseline on the same 40 interviews, hand-code 30 sources, and score theme coverage and quote accuracy.
The pipeline passes 24 backend tests on real Postgres + pgvector. The model-vs-baseline run is next, and its numbers go here once measured.
07What's next
- Run the Claude-vs-heuristic eval and publish the table
- Clip-level demo on recorded meetings (AMI corpus)
- Deploy, then record the demo