Engineering note
Answers that can show their source
Three of my products hand real records to AI: Ascend’s knowledge graph of lifting research, Zoning Signal’s archive of Central Florida planning decisions, and System Pulse’s readings from a Windows PC. Each is built to keep the source attached from the recorded claim to the agent that reads it.
- 01RecordEach claim carries the quote it came from.
- 02RetrieveLong and short records are searched differently.
- 03MeasureRetrieval changes are measured before they ship.
- 04ServeAgents read the same records, with sources attached.
01 · Record
Each claim carries the quote it came from.
Ascend’s knowledge graph holds about ten thousand claims distilled from public recordings by evidence-first researchers and coaches. Each claim has to carry at least one quotation that appears word for word in the transcript, with a link to the moment it was said. A claim without an anchor like that doesn’t ship. An audit checks every quotation against the transcript text and fails if any one is missing.
Six extraction passes read each recording independently. A curator merges what they find, and a verifier told to refute keeps, repairs or drops each claim against the transcript. The surviving claims sit in two tiers, the harvested claims and the syntheses the system writes across sources, linked by eight kinds of typed edge. When a synthesis is replaced, the old one is marked superseded and kept.
Zoning Signal works from public records. Its readings cite the agenda items, staff reports and minutes behind each signal, and I review the citations before anything publishes. The records also compile into an entity graph of 162 nodes and 479 typed edges in three evidence layers, and each edge names the dossier that states it.
- In productionAscendAnchored claims, an audit that fails on any missing quotation, and replaced syntheses kept as superseded.
- In productionZoning SignalCited readings with human review, and an entity graph with typed evidence layers.
02 · Retrieve
Long and short records are searched differently.
A one-line claim and a two-page synthesis need different treatment. One vector for a long synthesis averages everything it says, so a question about a single paragraph can miss it. In Ascend, short claims get one vector built from their title, statement, detail, conditions and takeaway. Long syntheses are also split into paragraphs that each keep the title, and a synthesis scores as its best paragraph. The vectors come from Voyage’s voyage-4 at 1024 dimensions and live in Postgres with pgvector, stored at half precision.
I built an approximate nearest-neighbour (HNSW) index first. Filtered to one tier of the graph, it returned nothing for that tier. So the matcher scans every vector exactly, behind a query barrier that keeps the database off the index. At ten thousand vectors that took about 80 milliseconds with perfect recall, and the index stays built for when the corpus gets too big for an exact scan. Row-level security is forced on every vector table with no policies, so the vectors can only be reached through a few database functions that return ids and scores.
Zoning Signal embeds each record whole and searches in tiers. A title containing the query ranks first, then close matches by meaning, then passages containing the words. I set it up that way after a measurement. For a short query naming something specific, the right record scored 0.287 on similarity and an unrelated sentence scored 0.274. That gap is too small to rank on.
- In productionAscendvoyage-4 at 1024 dimensions, per-tier representation, an exact scan behind a barrier, vectors reachable only through definer functions.
- In productionZoning SignalWhole-record embeddings and a tiered search designed from a measured margin.
03 · Measure
Retrieval changes are measured before they ship.
Ascend’s benchmark is 379 questions, written by a language model for 194 target claims, each with one right answer, so no judge is needed. The score is mean reciprocal rank: 1 if the right claim comes first, 0.5 if it comes second, and so on, averaged over the questions. On the synthesis tier, scoring by best paragraph reached 0.356 against 0.137 for one vector per claim, and put the right claim in the top fifty for 90.5 per cent of questions against 81. Every comparison is reported with a paired 95 per cent confidence interval.
I also tried reciprocal rank fusion offline, to combine rankings from different ways of embedding the claims. It adds up one over each result’s rank plus a constant across the rankings. It scored highest on paper, but the difference was within the noise, so I left it out of the product.
My adaptive rank fusion research studies fusion directly. It runs in numbered cycles, twenty-three so far, each with its protocol committed before the results come in and its metrics checked against NIST’s trec_eval. An early confidence-weighted fusion improved the top-ten metric with four search methods, and adding a fifth reversed it. The failed runs are published with the ones that held.
- All questions n = 379+0.096 +0.069 to +0.122
- Synthesis targets n = 188+0.218 +0.173 to +0.263
- Synthesis, detail only n = 93+0.261 +0.195 to +0.328
- Source claims n = 191-0.025 -0.040 to -0.010
- In productionAscendA known-item benchmark of 379 questions behind the matcher that shipped.
- ResearchAdaptive rank fusionPre-registered cycles with trec_eval-checked metrics, the reversals and failures included.
04 · Serve
Agents read the same records, with sources attached.
Zoning Signal publishes a Model Context Protocol server, the standard interface AI assistants use to call outside tools, and it is listed in the official registry. It is public and read-only, apart from one tool that takes feedback. An agent can look up a place, a meeting or a pattern, search by meaning, and trace how two named things connect through the entity graph. The trace finds every shortest route between them and ranks the routes by their most connected stop, lowest first. The server runs as its own Cloudflare Worker with the corpus compiled in, separate from the website.
Ascend’s graph has one endpoint. It embeds a question, retrieves candidate claims and reranks them for relevance, returning up to 24 claims with attribution, conditions and evidence, laid out to paste into a model’s context. It also says whether a claim directly addresses the question, and marks that judgment unavailable if the relevance stage cannot run. Nothing in the app calls it yet, and no model writes answers in Ascend. That part is what I’m building next.
System Pulse does this for a single PC. The desktop dashboard, a phone on a private network and local AI agents all read the same machine evidence through one service. Its MCP interface lets agents ask for the same readings, and each answer separates a reading that came back empty from one that could not be taken, so an agent can tell a clean result from a failed check.
- In productionZoning SignalA public, read-only MCP server in the official registry, with graph tracing between named things.
- In productionAscendA deployed endpoint that returns ranked, attributed claims ready for a model’s context; no app surface calls it yet.
- In developmentSystem PulseOne service behind the desktop, the phone and local agents, with typed outcomes for every reading.
Limits
Where the evidence stops.
These measurements are each project’s own, taken on its own benchmark, and none is an independent evaluation. Ascend’s questions were written by a language model for known targets. They test whether retrieval finds a known claim, and leave the quality of a finished answer untested. Zoning Signal’s thresholds are calibrations against its live index. The rank fusion work is exploratory research on small public collections, and its page says what each result does and doesn’t establish.