{"arm": "bitemporal", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 11.08, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"Legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-801 valid on 2026-03-01; superseded by D-802 on 2026-06-01\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "Legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": false, "note": "D-801 valid on 2026-03-01; superseded by D-802 on 2026-06-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 13.62, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid 2025-01-01 to 2026-03-31, superseded by D-102 from 2026-04-01\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 valid 2025-01-01 to 2026-03-31, superseded by D-102 from 2026-04-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 16.13, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"Support\",\n \"source_id\": \"SUP-POL-01\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Version 1 in force 2025-03-01 to 2026-01-14, superseded 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "Support", "source_id": "SUP-POL-01", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1 in force 2025-03-01 to 2026-01-14, superseded 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 16.55, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Authoritative definition from support, valid from 2026-01-15 onward.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "Authoritative definition from support, valid from 2026-01-15 onward."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 18.93, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 prevails per Finance ownership; later superseded by D-102\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "D-101", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "D-101 prevails per Finance ownership; later superseded by D-102"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 20.18, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-402\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Support's definition (D-402, valid 2026-01-15) prevailed over Engineering's concurrent definition per concept ownership rule.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "D-402", "policy_id": "CP-ORG", "unanswerable": false, "stale": false, "note": "Support's definition (D-402, valid 2026-01-15) prevailed over Engineering's concurrent definition per concept ownership rule."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 20.63, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-101 authoritative definition in force 2025-01-01 to 2026-03-31. Conflicting D-103 was non-authoritative.\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": "D-101", "policy_id": "CP-ORG", "unanswerable": false, "stale": false, "note": "D-101 authoritative definition in force 2025-01-01 to 2026-03-31. Conflicting D-103 was non-authoritative."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 9.02, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-801 was superseded by D-802 on 2026-06-01\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-801 was superseded by D-802 on 2026-06-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 19.44, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 definition excluded one-off services only, not notice period customers.\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 definition excluded one-off services only, not notice period customers."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 23.03, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"support\",\n \"source_id\": null,\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Support definition from 2026-01-15; engineering had different active definition\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": null, "policy_id": null, "unanswerable": false, "stale": false, "note": "Support definition from 2026-01-15; engineering had different active definition"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 15.77, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-401\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-401 was authoritative on 2025-12-01; superseded by D-402 from 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "support", "source_id": "D-401", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "D-401 was authoritative on 2025-12-01; superseded by D-402 from 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 16.81, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"an outage breaching the uptime term of a signed contract\",\n \"owner_team\": \"engineering\",\n \"source_id\": null,\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Engineering definition active since 2025-11-01; Support had concurrent definition from 2025-03-01\"\n}\n```\n", "obj": {"answer": "an outage breaching the uptime term of a signed contract", "owner_team": "engineering", "source_id": null, "policy_id": null, "unanswerable": false, "stale": true, "note": "Engineering definition active since 2025-11-01; Support had concurrent definition from 2025-03-01"}, "parsed": 1, "answer_correct": 0, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 21.0, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid on 2026-02-01, excluded one-off services. Superseded by D-102.\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 valid on 2026-02-01, excluded one-off services. Superseded by D-102."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 19.26, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"No, the definition did not exclude customers in their notice period\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-101 valid on 2025-12-31; D-103 conflicted; Finance prevailed. Notice period exclusion added in D-102 from 2026-04-01.\"\n}\n```\n", "obj": {"answer": "No, the definition did not exclude customers in their notice period", "owner_team": "Finance", "source_id": "D-101", "policy_id": "CP-ORG", "unanswerable": false, "stale": false, "note": "D-101 valid on 2025-12-31; D-103 conflicted; Finance prevailed. Notice period exclusion added in D-102 from 2026-04-01."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 12.14, "context_chars": 2694, "raw": "```json\n{\"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\", \"owner_team\": \"support\", \"source_id\": null, \"policy_id\": null, \"unanswerable\": false, \"stale\": false, \"note\": \"Definition from 2026-01-15 was current on query date 2026-01-20\"}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": null, "policy_id": null, "unanswerable": false, "stale": false, "note": "Definition from 2026-01-15 was current on query date 2026-01-20"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 14.55, "context_chars": 10621, "raw": "```json\n{\"answer\": \"customer records are kept for seven years after the contract ends\", \"owner_team\": \"Legal\", \"source_id\": \"D-801\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Valid until 2026-05-31; later superseded by D-802 and D-803\"}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "Legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "Valid until 2026-05-31; later superseded by D-802 and D-803"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 48.99, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Engineering also defined S1 via D-403 on that date\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "Engineering also defined S1 via D-403 on that date"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 14.23, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"finance\",\n \"source_id\": \"2025-01-01\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Definition was updated 2026-04-01 to exclude notice period customers.\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "finance", "source_id": "2025-01-01", "policy_id": null, "unanswerable": false, "stale": true, "note": "Definition was updated 2026-04-01 to exclude notice period customers."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 18.83, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-401\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Superseded by D-402 effective 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "Support", "source_id": "D-401", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "Superseded by D-402 effective 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 11.5, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-801 was valid through 2026-05-31; superseded by D-802 and D-803\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-801 was valid through 2026-05-31; superseded by D-802 and D-803"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 12.44, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-402\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-402 (Support) superseded D-401; D-403 (Engineering) also valid but Support owns the concept.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "D-402", "policy_id": "CP-ORG", "unanswerable": false, "stale": false, "note": "D-402 (Support) superseded D-401; D-403 (Engineering) also valid but Support owns the concept."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 15.43, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 was active on 2025-12-31; notice period exclusion added by D-102 on 2026-04-01\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "Finance", "source_id": "D-101", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "D-101 was active on 2025-12-31; notice period exclusion added by D-102 on 2026-04-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 14.97, "context_chars": 2694, "raw": "```json\n{\"answer\": \"No. The definition on 2025-12-31 excluded only one-off services, not customers in notice period.\", \"owner_team\": \"finance\", \"source_id\": null, \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Superseded by 2026-04-01 finance definition which added notice period exclusion.\"}\n```\n", "obj": {"answer": "No. The definition on 2025-12-31 excluded only one-off services, not customers in notice period.", "owner_team": "finance", "source_id": null, "policy_id": null, "unanswerable": false, "stale": true, "note": "Superseded by 2026-04-01 finance definition which added notice period exclusion."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 14.88, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"2024-01-01\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Superseded 2026-06-01; legal changed to 5 years, support to 7 years\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "2024-01-01", "policy_id": null, "unanswerable": false, "stale": true, "note": "Superseded 2026-06-01; legal changed to 5 years, support to 7 years"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 11.17, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid on 2025-12-31, excluded only one-off services. D-102 superseded it.\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "Finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 valid on 2025-12-31, excluded only one-off services. D-102 superseded it."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 39.29, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"Two active definitions: Support (D-401): 'outage affecting more than one customer with no workaround'. Engineering (D-403): 'outage breaching the uptime term of a signed contract'.\",\n \"owner_team\": null,\n \"source_id\": \"D-401; D-403\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Conflicting definitions from two teams were both valid on this date\"\n}\n```\n", "obj": {"answer": "Two active definitions: Support (D-401): 'outage affecting more than one customer with no workaround'. Engineering (D-403): 'outage breaching the uptime term of a signed contract'.", "owner_team": null, "source_id": "D-401; D-403", "policy_id": null, "unanswerable": false, "stale": false, "note": "Conflicting definitions from two teams were both valid on this date"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 56.82, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"D-101\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Sales definition D-103 conflicts; Finance policy D-101 excluded one-off services\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "D-101", "policy_id": "D-101", "unanswerable": false, "stale": false, "note": "Sales definition D-103 conflicts; Finance policy D-101 excluded one-off services"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 11.08, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"SUP-POL-03\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Version 2 effective 2026-01-15; Version 1 expired 2026-01-14.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "SUP-POL-03", "policy_id": null, "unanswerable": false, "stale": false, "note": "Version 2 effective 2026-01-15; Version 1 expired 2026-01-14."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 16.77, "context_chars": 6474, "raw": "```json\n{\"answer\": \"No\", \"owner_team\": \"finance\", \"source_id\": \"D-101\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"D-101 active on 2025-12-31 excluded only one-off services. D-102 introduced notice period exclusion from 2026-04-01.\"}\n```\n", "obj": {"answer": "No", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 active on 2025-12-31 excluded only one-off services. D-102 introduced notice period exclusion from 2026-04-01."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 21.43, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": null,\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Superseded by 2026-04-01 definition which added exclusion of customers in notice\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": null, "policy_id": null, "unanswerable": false, "stale": true, "note": "Superseded by 2026-04-01 definition which added exclusion of customers in notice"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 13.49, "context_chars": 6618, "raw": "```json\n{\"answer\": \"one-off services\", \"owner_team\": \"Finance\", \"source_id\": \"FIN-POL-04\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Version 1 was current on 2026-02-01, superseded by Version 2 on 2026-04-01.\"}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "FIN-POL-04", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1 was current on 2026-02-01, superseded by Version 2 on 2026-04-01."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 15.41, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"Support\",\n \"source_id\": \"SUP-POL-01\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Support Version 1 prevailed over Engineering's concurrent definition. Superseded 2026-01-15.\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "Support", "source_id": "SUP-POL-01", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "Support Version 1 prevailed over Engineering's concurrent definition. Superseded 2026-01-15."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 13.76, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"7 years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": null,\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Policy changed to 5 years (legal) effective 2026-06-01\"\n}\n```\n", "obj": {"answer": "7 years after the contract ends", "owner_team": "legal", "source_id": null, "policy_id": null, "unanswerable": false, "stale": true, "note": "Policy changed to 5 years (legal) effective 2026-06-01"}, "parsed": 1, "answer_correct": 0, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 1, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 26.39, "context_chars": 6474, "raw": "```json\n{\"answer\": \"an outage affecting more than one customer with no workaround\", \"owner_team\": \"support\", \"source_id\": \"D-401\", \"policy_id\": null, \"unanswerable\": false, \"stale\": false, \"note\": \"Engineering provided alternative definition from 2025-11-01\"}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "support", "source_id": "D-401", "policy_id": null, "unanswerable": false, "stale": false, "note": "Engineering provided alternative definition from 2025-11-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 14.08, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"Legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-801 valid on 2026-03-01; superseded 2026-06-01\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "Legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": false, "note": "D-801 valid on 2026-03-01; superseded 2026-06-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 11.45, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"No. On 2025-12-31, the authoritative ARR definition (D-101, valid 2025-01-01 to 2026-03-31) excluded only one-off services, not customers in notice period. The notice period exclusion was added in D-102 effective 2026-04-01.\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 superseded by D-102 from 2026-04-01\"\n}\n```\n", "obj": {"answer": "No. On 2025-12-31, the authoritative ARR definition (D-101, valid 2025-01-01 to 2026-03-31) excluded only one-off services, not customers in notice period. The notice period exclusion was added in D-102 effective 2026-04-01.", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 superseded by D-102 from 2026-04-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 11.83, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Authoritative definition from Support, effective 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "Authoritative definition from Support, effective 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 12.34, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-801 was superseded by D-802 on 2026-06-01\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-801 was superseded by D-802 on 2026-06-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 61.36, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": \"2025-01-01\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Finance definition excluded one-off services; sales definition included them.\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": "2025-01-01", "policy_id": null, "unanswerable": false, "stale": false, "note": "Finance definition excluded one-off services; sales definition included them."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 13.7, "context_chars": 6618, "raw": "```json\n{\"answer\": \"customer records are kept for seven years after the contract ends\", \"owner_team\": \"Legal\", \"source_id\": \"LEG-POL-01\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Version 1, superseded 2026-06-01 by LEG-POL-04\"}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "Legal", "source_id": "LEG-POL-01", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1, superseded 2026-06-01 by LEG-POL-04"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 32.25, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid through 2026-03-31; D-102 superseded it\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "Finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 valid through 2026-03-31; D-102 superseded it"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 29.72, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Concurrent conflicting definition from engineering (D-403). Version 2 supersedes version 1.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "Concurrent conflicting definition from engineering (D-403). Version 2 supersedes version 1."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 45.72, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 superseded by D-102 on 2026-04-01; D-103 from sales contradicts\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 superseded by D-102 on 2026-04-01; D-103 from sales contradicts"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 12.43, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Record D-801 was superseded on 2026-06-01 by D-802 and D-803\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "Record D-801 was superseded on 2026-06-01 by D-802 and D-803"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 12.64, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-401\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Definition superseded by D-402 from 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "Support", "source_id": "D-401", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "Definition superseded by D-402 from 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 19.37, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"an outage affecting more than one customer with no workaround\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-401\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Support's authoritative definition. Superseded by D-402 on 2026-01-15.\"\n}\n```\n", "obj": {"answer": "an outage affecting more than one customer with no workaround", "owner_team": "support", "source_id": "D-401", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "Support's authoritative definition. Superseded by D-402 on 2026-01-15."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-01", "family": "HIST", "rep": 1, "latency_s": 20.51, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid 2025-01-01 to 2026-03-31, superseded by D-102\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 valid 2025-01-01 to 2026-03-31, superseded by D-102"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 12.19, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"FIN-POL-04\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Version 1 excluded one-off services only. Version 2 added notice period exclusion from 2026-04-01.\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "Finance", "source_id": "FIN-POL-04", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1 excluded one-off services only. Version 2 added notice period exclusion from 2026-04-01."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-02", "family": "HIST", "rep": 0, "latency_s": 10.61, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"Legal\",\n \"source_id\": \"LEG-POL-01\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"Legal v1 active on 2026-03-01, superseded 2026-06-01 by v2\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "Legal", "source_id": "LEG-POL-01", "policy_id": null, "unanswerable": false, "stale": true, "note": "Legal v1 active on 2026-03-01, superseded 2026-06-01 by v2"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-04", "family": "HIST", "rep": 0, "latency_s": 67.58, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-403 from engineering concurrently provided conflicting definition; no precedence rule found.\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "D-403 from engineering concurrently provided conflicting definition; no precedence rule found."}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-05", "family": "HIST", "rep": 1, "latency_s": 17.93, "context_chars": 6618, "raw": "```json\n{\"answer\": \"No, only one-off services were excluded\", \"owner_team\": \"Finance\", \"source_id\": \"FIN-POL-04\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Version 1 superseded by Version 2 on 2026-04-01\"}\n```\n", "obj": {"answer": "No, only one-off services were excluded", "owner_team": "Finance", "source_id": "FIN-POL-04", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1 superseded by Version 2 on 2026-04-01"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 40.03, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"Support defined it as 'an outage affecting more than one customer with no workaround'; Engineering defined it as 'an outage breaching the uptime term of a signed contract'\",\n \"owner_team\": \"support, engineering\",\n \"source_id\": \"D-401, D-403\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Two concurrent conflicting definitions from different teams\"\n}\n```\n", "obj": {"answer": "Support defined it as 'an outage affecting more than one customer with no workaround'; Engineering defined it as 'an outage breaching the uptime term of a signed contract'", "owner_team": "support, engineering", "source_id": "D-401, D-403", "policy_id": null, "unanswerable": false, "stale": false, "note": "Two concurrent conflicting definitions from different teams"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 10.14, "context_chars": 6618, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"Support\",\n \"source_id\": \"SUP-POL-03\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Version 2 definition in effect from 2026-01-15\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "Support", "source_id": "SUP-POL-03", "policy_id": null, "unanswerable": false, "stale": false, "note": "Version 2 definition in effect from 2026-01-15"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-05", "family": "HIST", "rep": 0, "latency_s": 12.73, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"No\",\n \"owner_team\": \"finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 active on 2025-12-31 excluded only one-off services, not notice-period customers.\"\n}\n```\n", "obj": {"answer": "No", "owner_team": "finance", "source_id": "D-101", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-101 active on 2025-12-31 excluded only one-off services, not notice-period customers."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dim_no_gov", "item": "HIST-04", "family": "HIST", "rep": 1, "latency_s": 35.98, "context_chars": 6474, "raw": "```json\n{\n \"answer\": \"an outage affecting any customer with no workaround, or any confirmed data loss\",\n \"owner_team\": \"support\",\n \"source_id\": \"D-402\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"D-403 from engineering also valid but D-402 is more recent\"\n}\n```\n", "obj": {"answer": "an outage affecting any customer with no workaround, or any confirmed data loss", "owner_team": "support", "source_id": "D-402", "policy_id": null, "unanswerable": false, "stale": false, "note": "D-403 from engineering also valid but D-402 is more recent"}, "parsed": 1, "answer_correct": 1, "owner_correct": 1, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 1, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "dimension", "item": "HIST-02", "family": "HIST", "rep": 1, "latency_s": 24.61, "context_chars": 11191, "raw": "```json\n{\n \"answer\": \"customer records are kept for seven years after the contract ends\",\n \"owner_team\": \"legal\",\n \"source_id\": \"D-801\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-801 was authoritative on 2026-03-01; superseded by D-802 on 2026-06-01.\"\n}\n```\n", "obj": {"answer": "customer records are kept for seven years after the contract ends", "owner_team": "legal", "source_id": "D-801", "policy_id": null, "unanswerable": false, "stale": true, "note": "D-801 was authoritative on 2026-03-01; superseded by D-802 on 2026-06-01."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "wiki", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 13.85, "context_chars": 6618, "raw": "```json\n{\"answer\": \"one-off services\", \"owner_team\": \"Finance\", \"source_id\": \"FIN-POL-04\", \"policy_id\": null, \"unanswerable\": false, \"stale\": true, \"note\": \"Version 1 superseded 2026-04-01 by Version 2\"}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "FIN-POL-04", "policy_id": null, "unanswerable": false, "stale": true, "note": "Version 1 superseded 2026-04-01 by Version 2"}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "graph_catalog", "item": "HIST-01", "family": "HIST", "rep": 0, "latency_s": 15.08, "context_chars": 10621, "raw": "```json\n{\n \"answer\": \"one-off services\",\n \"owner_team\": \"Finance\",\n \"source_id\": \"D-101\",\n \"policy_id\": \"CP-ORG\",\n \"unanswerable\": false,\n \"stale\": true,\n \"note\": \"D-101 valid until 2026-03-31; superseded by D-102. Finance owns ARR.\"\n}\n```\n", "obj": {"answer": "one-off services", "owner_team": "Finance", "source_id": "D-101", "policy_id": "CP-ORG", "unanswerable": false, "stale": true, "note": "D-101 valid until 2026-03-31; superseded by D-102. Finance owns ARR."}, "parsed": 1, "answer_correct": 1, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "bitemporal", "item": "HIST-03", "family": "HIST", "rep": 1, "latency_s": 45.15, "context_chars": 8932, "raw": "```json\n{\n \"answer\": \"an outage breaching the uptime term of a signed contract\",\n \"owner_team\": \"Engineering\",\n \"source_id\": \"D-403\",\n \"policy_id\": null,\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Valid from 2025-11-01 onwards\"\n}\n```\n", "obj": {"answer": "an outage breaching the uptime term of a signed contract", "owner_team": "Engineering", "source_id": "D-403", "policy_id": null, "unanswerable": false, "stale": false, "note": "Valid from 2025-11-01 onwards"}, "parsed": 1, "answer_correct": 0, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0} {"arm": "vector_memory", "item": "HIST-03", "family": "HIST", "rep": 0, "latency_s": 66.14, "context_chars": 2694, "raw": "```json\n{\n \"answer\": \"an outage breaching the uptime term of a signed contract\",\n \"owner_team\": \"engineering\",\n \"source_id\": \"2025-11-01\",\n \"policy_id\": \"2025-11-01\",\n \"unanswerable\": false,\n \"stale\": false,\n \"note\": \"Engineering definition from 2025-11-01 was most recent; support's conflicting definition from 2025-03-01\"\n}\n```\n", "obj": {"answer": "an outage breaching the uptime term of a signed contract", "owner_team": "engineering", "source_id": "2025-11-01", "policy_id": "2025-11-01", "unanswerable": false, "stale": false, "note": "Engineering definition from 2025-11-01 was most recent; support's conflicting definition from 2025-03-01"}, "parsed": 1, "answer_correct": 0, "owner_correct": 0, "policy_correct": 0, "source_correct": 0, "abstained": 0, "false_abstention": 0, "stale_correct": 0, "overclaim": 0, "fabricated": 0, "scored_answer": 1, "scored_owner": 0, "scored_policy": 0, "scored_source": 0, "scored_abst": 0, "scored_stale": 0}