Zoning Signal

Harvesting, publishing and search

Twenty-five jurisdictions post their planning agendas and minutes in eight different ways, from vendor portals such as CivicClerk and Laserfiche to a city’s own pages and bare PDFs, each in its own shape. Everything below turns that scatter into one readable set of files, the same ones the build checks and the agent server queries. The aim is that every figure on the site comes from the build.

Harvest

CivicClerk REST API · Laserfiche WebLink · Playwright Chromium · pdfplumber · tesseract at 300 DPI

Python tools pull agendas and minutes from where each city posts them and extract the text, reading scanned pages with tesseract at 300 DPI.

How it works

Python tools pull agendas and minutes from where each city posts them, the CivicClerk API for meeting metadata, the Laserfiche WebLink API page by page and headless Chromium for portals that are JavaScript applications; pdfplumber extracts the text, and when a scanned packet yields none, each page is rendered at 300 DPI for tesseract to read. Of the 376 meeting readings in the build of 2026-09-21, 360 name their platform, all 355 harvested records among them: CivicClerk 153, CivicEngage 54, city websites 52, direct PDFs 45, Laserfiche 29, MuniCode Meetings 16, AV Capture All 9, Granicus 2.

Corpus

YAML front-matter validator · SHA-256 per document · content-addressed version · fails on duplicate (type, slug)

An agent writes one Markdown record per meeting, a validator refuses malformed records, and one build step compiles, hashes and names the whole corpus by its own hash.

How it works

An agent reads the extracted text and writes one Markdown record per meeting from a template; a validator refuses a record that lacks one of eight front-matter fields, uses a value outside its enumeration or misses one of three required sections. One build step then compiles every Markdown and MDX source into a single artifact, hashes each document, names the corpus by the first sixteen hex characters of its own hash, and fails on malformed front matter or a duplicate (type, slug).

Entity graph

162 nodes, 479 edges · three evidence layers · authored_on provenance · byte-compare gate · 13 golden fixtures

A second build step compiles the entity dossiers into a typed graph of 162 nodes and 479 edges, and each edge records which dossier states it.

How it works

A second build step reads the front matter of the 69 entity dossiers and compiles a typed graph of 78 meetings, 69 entities and 15 named patterns, joined by 479 edges in three layers, where a dossier places the actor in a meeting record (250), names a related entity (169) or says it exhibits a pattern (60), and each edge records which dossier states it. Geographic and editorial links were excluded as hops by measurement (in the full front-matter graph one corridor node carries 118 links and 94.5% of 2,346 entity pairs sit within two steps, so a route through them would distinguish nothing); verify:graph rebuilds the graph in memory and byte-compares it with the emitted file, and 13 golden queries pin what trace answers.

Graph query

trace_connection · BFS · shortest-path DAG · minimax bottleneck DP · base rates for 7 layer subsets

trace_connection finds every shortest route between two named things and ranks a route through a hub below one through rarely linked nodes.

How it works

trace_connection runs a breadth-first search from one named thing to another, counts every shortest route through the predecessor DAG, and computes for each route the degree of its busiest stop; it ranks by that degree, lowest first, then by hops, then by how many equally short routes exist, so a route through a hub ranks below one through rarely linked nodes. The build measures base rates for all seven combinations of the evidence layers (component counts, degree profiles, distance and uniqueness histograms), and every answer ships the rates for the layers it traversed, with its own reach: 78 of 376 meeting readings across 11 of 25 jurisdictions.

Retrieval

voyage-4, 1024 dimensions · pgvector halfvec(1024) · Postgres full-text, simple config · weighted reciprocal rank fusion, k = 3 · held-out MRR 0.683 to 0.887 · scope and evidence

A search ranks every record three ways, by meaning, by the words in its text and by its title, and fuses the lists; on 222 held-out questions, mean reciprocal rank rose from 0.683 to 0.887.

How it works

Each record is embedded whole, title, subtitle and body up to 32,000 characters, with voyage-4 and stored as a half-precision pgvector column. One database function, run as its caller, ranks the published rows three ways to a depth of 50: exact cosine against the query’s embedding; Postgres full-text search in the simple configuration, which keeps an identifier such as 2026-021R as written, over title and subtitle weighted above the body; and the same search over title and subtitle alone. Weighted reciprocal rank fusion (k = 3; weights 1, 1.25 and 3, taken from the middle of the development plateau) merges the lists, and every hit reports its true cosine and its rank in each. On a known-item benchmark of 524 questions split by answer record, tuned on 302 and frozen before the other 222 were read, mean reciprocal rank rose from 0.683 for the previous precedence cascade over the same complete index to 0.887, a paired difference of +0.204 (95% CI 0.154 to 0.254), 68 questions better and 6 worse. Identifiers rose from 0.326 to 0.840 and proper nouns from 0.402 to 0.905. Against the cascade with its one threshold tuned on the development split (0.816), fusion leads by +0.071 (0.032 to 0.109). Every response states its scope (a covered place, a place outside coverage, or no place it recognises) and its evidence, strong when one of the top three records contains the query’s terms or resembles it at cosine 0.5 or above.

Agent server

MCP 2025-11-25 · stateless Streamable HTTP · com.zoningsignal/observatory · tool annotations, outputSchema · resources, prompts, completion · Worker with the corpus inlined

The MCP server is one stateless handler in a Cloudflare Worker with the corpus and graph bundled in, listed on the official registry since April 2026.

How it works

The server is one Request-in, Response-out handler in its own Cloudflare Worker, with corpus.json and graph.json bundled as modules through esbuild and a build that refuses to bundle them if they disagree on document count; sixteen of the 18 tools its discovery card listed at version 2.3.0 on 2026-09-26 answer from that inlined corpus, semantic search calls Voyage and Postgres, and the feedback tool writes one row. It speaks JSON-RPC over stateless streamable HTTP, is listed on the official registry through eleven versions since April 2026, and uses the whole protocol surface: every tool carries annotations (all closed-world; all but the feedback tool read-only and idempotent) and an output schema, beside resources, resource templates, prompts, argument completion and logging.

Security and telemetry

RLS on every table · SECURITY INVOKER search · anon key · SECURITY DEFINER aggregates · salted SHA-256 IP hash · ok_empty

Row-level security is enabled on every table, and each message writes one log row with the client address kept only as a salted hash.

How it works

Row-level security is enabled on every table: the public reads published rows only, and the vector search function runs as its caller, with the anon key, so that rule still applies inside it; the invocation and feedback logs have no public read policy, and the agent observatory reads them only through definer functions, five returning aggregates and one returning feedback an operator marked for publication. Each message writes one row after the response is sent, with the client address kept only as a salted SHA-256 hash truncated to 32 characters, and a success that returned nothing, or only a weak match, recorded as ok_empty, so demand the corpus does not yet meet is kept on record.

Limits. The verify gates for the graph, the trace fixtures, the URL contract and the live site are run by hand. The vector mirror is synced by an operator, outside the build, so semantic search reads what was last synced, which can trail the corpus as built. The benchmark’s questions are the project’s own: identifiers and names mined from the records, paraphrases written by a model, and 29 logged agent queries labelled by hand. The title list was added after some test questions had been seen, so its share of the gain is partly in-sample; without it, fusion scores 0.863. Most of the gain comes from the two text lists. On paraphrase alone, fusion trails the cascade (0.911 to 0.880 on the test split, 0.927 to 0.909 on 66 fresh paraphrases), a difference that is not significant. The scope gazetteer and the evidence threshold were set against the logged queries they are scored on. Latency was measured locally, not in production. Citation review is human. At the September source checkpoint the graph linked 78 of 376 then-published meeting readings in 11 of 25 jurisdictions. Confidence grades are assessed by the operator, not computed, and the track record observed on 2026-10-16 holds eight resolved calls, too few for a percentage.

Code read at the observatory repository on 2026-09-20 and the harvest repository on 2026-09-16; counts recomputed from the corpus build of 2026-09-21; registry listing and discovery card fetched 2026-09-26; retrieval measured in the observatory’s benchmark on 2026-09-26.