# Vercy memory retrieval benchmark v1

This is a small, reproducible paired test, not a universal claim about memory.
It compares the same fresh language model answering the same 15 questions from
the same synthetic Northstar facts under two delivery conditions:

1. **Vercy retrieval:** the deterministic local index selects typed records,
   relations, events and bitemporal facts for each question;
2. **flat notes:** the model receives the same facts as chronological prose in
   one document among 1,000 deterministic, irrelevant equipment notes.

Each condition is run in a fresh non-persistent session with tools disabled.
Question order, prompt and answer format are fixed. Exact answers are normalized
for case and punctuation and matched against the published acceptable values.
The runner stores the model/CLI metadata, raw synthetic answers, corpus digest,
per-question score and aggregate score. It never stores credentials or private
Dimension data.

This evaluates retrieval-supported question answering at moderate distractor
scale. It does not evaluate vector search, long-term human adoption, safety,
legal truth or every model/provider. A result is evidence for this fixture and
model version only. Re-run after runtime, data or model changes; publish a new
result rather than replacing an old claim silently.