Vercy · research

Research.

What structured memory does for an agent, measured rather than asserted. Negative results stay on the page.

2026-09-07

Tools enabled: the limitation every earlier study declared

Question: Every study in this series disabled tools and named that as its main limitation: with a shell and a data file the arithmetic component of the error changes, and the balance between representations might shift. Does it?
Finding: It shifts, against the common assumption. Conditions carrying complete definitions went to 97-100%; conditions without them did not move at all, 26.5% to 23.5% for schema alone. Worse, the rate of answering with exactly the number a superseded rule produces rose from 16.2% to 29.4%, and appropriate abstention fell from 83.3% to 50.0%. Tools do not substitute for knowing what the words mean; they raise the cost of not knowing, and they make the failure quieter.

Report Instrument Raw runs

2026-09-07

Collaborative Memory Benchmark

Question: When six teams write into one shared memory, does it matter whether ownership, the conflict rule and the release list live inside the record or in a governance layer beside it? Compared against reimplementations of three shipping product shapes.
Finding: Where it lives did not matter; whether it exists mattered a lot. Governance inside the record scored 95.2%, the same graph with a separate catalogue 93.5%, a plain team wiki 96.8%, all pairwise p = 1.00. But adding governance to a bi-temporal graph took it from 77.4% to 93.5%, p = 0.039, and every representation carrying release rules made all 60 cross-team disclosure decisions correctly while the graph without them made two critical over-shares and withheld 41% of what teams were entitled to see.

Report Methods Instrument Raw runs

2026-09-06

Personal Dimension Benchmark

Question: For one person's own records, what does the representation do for the assistant reading them, and which part of a structured representation carries which capability?
Finding: Structure did not beat complete notes: chronological notes carrying every fact, date and source scored 93.4% against 100% for a structured dimension, three items apart, p = 0.25. Every ablation was decisive. Without validity intervals, historical accuracy fell from 100% to 0% and expiry judgement to 20%, with current-state questions untouched. Without provenance, conflict resolution fell from 100% to 33%. Each field class is the sole support of one capability.

Report Methods Instrument Raw runs

2026-09-06

Semantic Grounding Benchmark

Question: Does a versioned definition layer make an agent more accurate when two organisations define the same words differently?
Finding: No. Continuous prose carrying the same facts matched versioned records exactly, 76.5% against 76.5%, p = 1.00: accuracy follows completeness, not representation. The structured conditions were the only ones that cited the rule applied, located a missing definition and its owner, routed a request to the right owner, and made 72 disclosure decisions without a critical error.

Report Methods External review Instrument Raw runs

2026-08

Memory retrieval

Question: Does retrieval over a structured dimension beat flat notes on recall of a project's own history?
Finding: Parity on accuracy: all 15 questions answered correctly under both conditions across three fresh runs. Retrieval supplied 57.9% less context. The result is a tie, and is published as one.

Report Methods Raw runs

How these are published

Every study here ships the instrument that produced it, the unedited raw runs of every model call, and the record of any review that invalidated an earlier version. Where a claim did not survive the data, the claim was changed and the data was not. A benchmark whose failures are not published is a marketing artefact rather than a measurement.