{"schema":"https://ver.cy/schemas/card/1.0.0","id":"vr.wm-ai-013","code":"wm-ai-013-ablation-study","url":"https://ver.cy/models/wm-ai-013-ablation-study/","name":"Ablation Study","alternateNames":[],"kind":"world-model","status":"published","version":"0.1.0","language":"en","classifiers":{"family":"World Models","category":"Information and virtual systems","entryKind":"entity","plane":"","domain":["INF.AI.ABL"],"industry":["Cross-industry"],"navPath":"NAV.INF.AI.ABL","tags":["ablation","study","inf.ai.abl"],"facets":{}},"whatItIs":"One persistent, versioned ablation study record owns its protocol, arm comparison intent, evidence references and bounded interpretations. Runs, datasets, scorer implementations and benchmark result masters remain external. This is a proposed research specification, not an executed experiment.","purpose":"Record controlled with-factor and without-factor comparisons on a common evaluation corpus using one scorer contract per contrast.","scope":{"in":["Study identity, hypothesis, intervention arms and protocol revisions","Common evaluation cases, scorer contract, controlled changes and deviations","Run references, failures, paired observations, effect estimates and uncertainty","Claim qualification, review, evidence access and retention"],"out":["Executing training, inference, scoring or deployment","Mastering datasets, labels, model weights, scorer code or general benchmark leaderboards","Universal causal identification, statistical test selection or safety certification","Operational instructions for harmful evaluations"],"boundaries":[{"neighbor":"WM-AI-003 AI Model Evaluation","distinction":"Registry parent is a conceptual specialization lead. The study adds controlled intervention contrasts; no unverified runtime inheritance or containment is asserted."},{"neighbor":"WM-AI-004 AI Inference / Agent Run and WM-AI-006 Model Training / Fine-tuning Run","distinction":"Execution attempts have separate masters. A study can exist before runs and may reference optional training runs when retraining is part of its intervention."},{"neighbor":"WM-AI-009 Evaluation Dataset / Benchmark and WM-AI-014 Ground-truth Label Set","distinction":"Frozen case and label references are used; dataset curation and label lifecycle are not owned here."},{"neighbor":"WM-AI-011 Evaluation Harness and Scorer and WM-AI-012 Benchmark Result","distinction":"Scorer definitions and result records remain external. The study owns the rationale for comparing arms, not the master scores or implementation."},{"neighbor":"Reusable experiment design and factorial experiment","distinction":"This root is an instantiated study record, not a design pattern. Multi-arm contrasts are allowed with declared settings; a single ablation cannot isolate all interactions."}]},"distinguishingFeatures":["Unlike WM-AI-003 AI Model Evaluation: Registry parent is a conceptual specialization lead. The study adds controlled intervention contrasts; no unverified runtime inheritance or containment is asserted.","Unlike WM-AI-004 AI Inference / Agent Run and WM-AI-006 Model Training / Fine-tuning Run: Execution attempts have separate masters. A study can exist before runs and may reference optional training runs when retraining is part of its intervention.","Unlike WM-AI-009 Evaluation Dataset / Benchmark and WM-AI-014 Ground-truth Label Set: Frozen case and label references are used; dataset curation and label lifecycle are not owned here.","Unlike WM-AI-011 Evaluation Harness and Scorer and WM-AI-012 Benchmark Result: Scorer definitions and result records remain external. The study owns the rationale for comparing arms, not the master scores or implementation.","Unlike Reusable experiment design and factorial experiment: This root is an instantiated study record, not a design pattern. Multi-arm contrasts are allowed with declared settings; a single ablation cannot isolate all interactions."],"structure":{"bundles":[{"id":"bundle-study","name":"Study charter and protocol","description":"Identify the study and preserve its declared intent.","layers":[{"id":"layer-identity","name":"Study identity","description":"Proposed record obligations for study identity.","findings":[{"id":"finding-identity","name":"Study identity and accountable scope","description":"One persistent study record binds its purpose and accountable role across protocol revisions. A run, report or reusable experimental recipe has a different identity.","questions":[{"text":"Which authoritative study identifier and revision distinguish this study from a rerun or replication?","id":"finding-identity-q01","kind":"identity"},{"text":"Which accountable study role owns the record and which evidence custodian maintains its references?","id":"finding-identity-q02","kind":"ownership"},{"text":"What precise factor contribution is being investigated on which target population?","id":"finding-identity-q03","kind":"definition"},{"text":"Is this a planned, exploratory or confirmatory study and how was that classification determined?","id":"finding-identity-q04","kind":"classification"}]}]},{"id":"layer-protocol","name":"Protocol and revision","description":"Proposed record obligations for protocol and revision.","findings":[{"id":"finding-protocol","name":"Frozen comparison intent","description":"A frozen protocol revision states intended contrasts, metrics and decision rules. Amendments remain visible and post-result choices cannot be relabelled as prespecified.","questions":[{"text":"When were hypotheses and analysis rules frozen relative to the first outcome inspection?","id":"finding-protocol-q01","kind":"temporal"},{"text":"Which transitions move this study from design through execution, analysis and review to closure or withdrawal?","id":"finding-protocol-q02","kind":"lifecycle"},{"text":"Which protocol amendments occurred and which comparisons must be repeated or reclassified?","id":"finding-protocol-q03","kind":"event"},{"text":"Who may freeze or amend the protocol and what review is required for an outcome-informed change?","id":"finding-protocol-q04","kind":"authority"}]}]}]},{"id":"bundle-design","name":"Interventions and controls","description":"Make arm changes and conditional comparison boundaries inspectable.","layers":[{"id":"layer-intervention","name":"Factor intervention","description":"Proposed record obligations for factor intervention.","findings":[{"id":"finding-intervention","name":"With-factor and without-factor arms","description":"Each contrast names a reference arm and an ablated arm. Removal, masking or replacement semantics are explicit; training-time changes and inference-time disabling answer different questions. Incidental changes limit attribution.","questions":[{"text":"Which component or configuration factor is removed, masked or replaced in each arm?","id":"finding-intervention-q01","kind":"composition"},{"text":"Does the intervention require retraining or only alter inference and what remains fixed?","id":"finding-intervention-q02","kind":"process"},{"text":"How is actual intervention compliance checked against intended arm configurations?","id":"finding-intervention-q03","kind":"validation"},{"text":"What happens when removing the factor makes an arm non-executable or changes its interface?","id":"finding-intervention-q04","kind":"exception"}]}]},{"id":"layer-controls","name":"Experimental controls","description":"Proposed record obligations for experimental controls.","findings":[{"id":"finding-controls","name":"Confounders and interaction limits","description":"The local design records nuisance variables, budget policy and blocks. A contrast at one background setting estimates only that conditional comparison; multiple removals do not automatically identify independent contributions.","questions":[{"text":"Which nuisance factors and execution budgets are controlled, blocked or left uncontrolled?","id":"finding-controls-q01","kind":"constraint"},{"text":"How are run order and repeat allocation determined within declared blocks?","id":"finding-controls-q02","kind":"process"},{"text":"Which other factor settings condition the contrast and which interactions are untested?","id":"finding-controls-q03","kind":"relationship"},{"text":"Could unequal tuning or resource allowances explain the observed difference?","id":"finding-controls-q04","kind":"quality"}]}]}]},{"id":"bundle-evaluation","name":"Common evaluation contract","description":"Bind the shared corpus and scorer before admitting comparisons.","layers":[{"id":"layer-corpus","name":"Corpus binding","description":"Proposed record obligations for corpus binding.","findings":[{"id":"finding-corpus","name":"Common evaluation cases and labels","description":"Both arms reference the same frozen evaluation case set, split and label revision for a contrast. Input transformation may be the declared intervention while case identity remains stable. Membership drift and training overlap are recorded as threats.","questions":[{"text":"Which dataset revision, split and ordered or keyed case membership are common to both arms?","id":"finding-corpus-q01","kind":"identity"},{"text":"Which label revision and preprocessing lineage support each evaluation case?","id":"finding-corpus-q02","kind":"provenance"},{"text":"What checks identify leakage, duplicate cases or exposure during tuning and training?","id":"finding-corpus-q03","kind":"quality"},{"text":"How are case omissions or label corrections handled without silently changing the paired cohort?","id":"finding-corpus-q04","kind":"exception"}]}]},{"id":"layer-scorer","name":"Scoring contract","description":"Proposed record obligations for scoring contract.","findings":[{"id":"finding-scorer","name":"One scorer definition per contrast","description":"A contrast binds one scorer version, rubric, metric direction and aggregation rule across arms. A stochastic scorer needs its own repeat policy. Changing a scorer creates a new scoring revision for both arms.","questions":[{"text":"Which metric, units, preferred direction and aggregation define the comparison score?","id":"finding-scorer-q01","kind":"measurement"},{"text":"Which harness, scorer code or rubric revision and configuration are shared by both arms?","id":"finding-scorer-q02","kind":"identity"},{"text":"How are scorer randomness, blinding and order effects handled?","id":"finding-scorer-q03","kind":"constraint"},{"text":"What evidence shows both arms were scored with the pinned scoring contract?","id":"finding-scorer-q04","kind":"validation"}]}]}]},{"id":"bundle-execution","name":"Execution evidence","description":"Connect actual attempts to complete, paired observations.","layers":[{"id":"layer-runs","name":"Run lineage","description":"Proposed record obligations for run lineage.","findings":[{"id":"finding-runs","name":"Execution realization and repeats","description":"Execution remains externally mastered. The study indexes arm assignments, run attempts, environment snapshots and repetition units, preserving differences between a retry, an independent repeat and a reanalysis.","questions":[{"text":"Which external training, inference and evaluation runs realize each arm and repeat?","id":"finding-runs-q01","kind":"relationship"},{"text":"Which environment, configuration, seed and dependency revisions were actually used?","id":"finding-runs-q02","kind":"provenance"},{"text":"When did each attempt start and end and which protocol revision governed it?","id":"finding-runs-q03","kind":"temporal"},{"text":"Which attempts are retries, independent repeats or rescoring of existing outputs?","id":"finding-runs-q04","kind":"classification"}]}]},{"id":"layer-observations","name":"Paired observations","description":"Proposed record obligations for paired observations.","findings":[{"id":"finding-observations","name":"Complete outcomes and failures","description":"Case and repeat keys join the two arms. Failures and absent results remain explicit; complete-case analysis must disclose exclusions and its population. A successful retry never overwrites a failed attempt.","questions":[{"text":"Which per-case or aggregate observations and missingness indicators back each arm score?","id":"finding-observations-q01","kind":"evidence"},{"text":"Do pair keys join one-to-one and how are duplicates, unmatched cases and nonfinite scores rejected?","id":"finding-observations-q02","kind":"validation"},{"text":"Which failures, timeouts and exclusions occurred and how do they affect the denominator?","id":"finding-observations-q03","kind":"exception"},{"text":"How are original attempts preserved when a retry or rescore produces a new observation?","id":"finding-observations-q04","kind":"retention"}]}]}]},{"id":"bundle-analysis","name":"Analysis and interpretation","description":"Keep estimates, uncertainty and conclusions distinguishable.","layers":[{"id":"layer-effect","name":"Effect estimation","description":"Proposed record obligations for effect estimation.","findings":[{"id":"finding-effect","name":"Signed comparison and uncertainty","description":"The proposed convention is delta = score_with_factor - score_without_factor. Metric direction controls interpretation. Uncertainty identifies its sampling unit and method; dataset-level test guidance is not automatically applicable to item-level observations.","questions":[{"text":"What signed difference and practical importance threshold are reported for each contrast?","id":"finding-effect-q01","kind":"measurement"},{"text":"Which variability sources and sampling units support the uncertainty estimate?","id":"finding-effect-q02","kind":"quality"},{"text":"Which interval or test procedure was selected and why are its assumptions appropriate?","id":"finding-effect-q03","kind":"process"},{"text":"How are undefined relative changes, insufficient repeats and non-estimable intervals represented?","id":"finding-effect-q04","kind":"exception"}]}]},{"id":"layer-interpretation","name":"Interpretation and robustness","description":"Proposed record obligations for interpretation and robustness.","findings":[{"id":"finding-interpretation","name":"Bounded conclusions and multiplicity","description":"The study separates observed differences from explanatory claims and deployment decisions. Multiple contrasts, metrics or selected slices require a disclosed analysis family and selection history; a null result alone does not establish equivalence.","questions":[{"text":"What conclusion is supported within this corpus, scorer and intervention setting?","id":"finding-interpretation-q01","kind":"decision"},{"text":"Which hypothesis family and stopping or selection rules bound multiplicity and selective reporting?","id":"finding-interpretation-q02","kind":"constraint"},{"text":"Which sensitivity checks assess exclusion, scoring or seed choices and where do conclusions change?","id":"finding-interpretation-q03","kind":"quality"},{"text":"What additional evidence is needed before claiming equivalence, causal isolation or wider generalization?","id":"finding-interpretation-q04","kind":"requirement"}]}]}]},{"id":"bundle-assurance","name":"Review and evidence governance","description":"Support review and controlled reuse of study evidence.","layers":[{"id":"layer-review","name":"Review and release","description":"Proposed record obligations for review and release.","findings":[{"id":"finding-review","name":"Evidence-backed review disposition","description":"A reviewer assesses the record and its limitations independently of execution status. Review can reject or qualify a conclusion. A local release records availability and reproducibility limits without certifying safety or external acceptance.","questions":[{"text":"Which qualified reviewer evaluated the contrast and what conflicts of interest were declared?","id":"finding-review-q01","kind":"authority"},{"text":"What review disposition and unresolved issues apply to each reported claim?","id":"finding-review-q02","kind":"state"},{"text":"Which evidence export profile and pinned mapping preserve links and disclose losses?","id":"finding-review-q03","kind":"interoperability"},{"text":"Which released artifacts permit recomputation and which dependencies remain unavailable?","id":"finding-review-q04","kind":"evidence"}]}]},{"id":"layer-governance","name":"Access and retention","description":"Proposed record obligations for access and retention.","findings":[{"id":"finding-governance","name":"Controlled evidence custody","description":"The proposed study policy separates public conclusions from restricted inputs and outputs. Disclosure, retention and disposal use local authority and rights checks. Dangerous evaluations are represented only by policy-level scope and oversight metadata.","questions":[{"text":"Who may view each bundle, layer, finding and artifact under the evidence access policy?","id":"finding-governance-q01","kind":"access"},{"text":"Which personal or sensitive content requires minimization or redaction before release?","id":"finding-governance-q02","kind":"privacy"},{"text":"Which retention schedule, preservation holds and disposal rules govern each evidence class?","id":"finding-governance-q03","kind":"retention"},{"text":"What authorization and audit trail govern access changes or restricted evaluation metadata?","id":"finding-governance-q04","kind":"security"}]}]}]}]},"agentConduct":{"may":["Register study intent: Proposed, unimplemented local operation: register study intent.","Freeze comparison protocol: Proposed, unimplemented local operation: freeze comparison protocol.","Link execution evidence: Proposed, unimplemented local operation: link execution evidence.","Check contrast comparability: Proposed, unimplemented local operation: check contrast comparability.","Record analysis and interpretation: Proposed, unimplemented local operation: record analysis and interpretation.","Record review and release decision: Proposed, unimplemented local operation: record review and release decision."],"mustNot":["Never overwrite original observations with retries.","Deny access unless owner policy grants a role and purpose at the relevant scope; public summaries do not imply public raw data."],"requiresHuman":[]},"ethics":{"considerations":["Restrict harmful evaluation details to policy-level purpose, authorization, oversight and disclosure decisions.","Adopters must supply applicable evidence rights, privacy, ethics and retention rules.","Operational instructions for harmful evaluations"],"affectedParties":[]},"owners":{"steward":"Accountable study steward role and delegated authority record.","roles":[{"name":"Study steward","responsibilities":["Own purpose, scope, protocol revisions and lifecycle decisions."]},{"name":"Experiment operator","responsibilities":["Supply external run and configuration evidence without changing the frozen protocol."]},{"name":"Analysis reviewer","responsibilities":["Assess pairing, assumptions, uncertainty and bounded claims; declare independence limits."]},{"name":"Evidence custodian","responsibilities":["Enforce access, integrity, retention and disclosure rules."]},{"name":"Release reviewer","responsibilities":["Review rights and unresolved holds before recording release eligibility."]}],"masterSystems":[]},"relations":[{"target":"WM-AI-003","type":"aligned","note":"Candidate conceptual specialization from registry parent; executable inheritance remains unverified."},{"target":"WM-AI-009","type":"references","note":"Candidate frozen evaluation corpus binding; required case identity is a study obligation, not a forced dependency on this package."},{"target":"WM-AI-011","type":"references","note":"Candidate pinned scorer and harness definition; no scorer code is mastered here."},{"target":"WM-AI-012","type":"references","note":"Candidate external result evidence for each arm or contrast."},{"target":"WM-AI-004","type":"references","note":"Optional external inference run receipts indexed by arm and repeat."},{"target":"WM-AI-006","type":"references","note":"Optional training run references for interventions involving retraining."},{"target":"WM-AI-014","type":"references","note":"Optional label set revision when the evaluation uses ground truth."},{"target":"https://www.w3.org/TR/prov-o/","type":"aligned","note":"Conceptual provenance mapping only; no implemented conformance claim."},{"target":"https://www.researchobject.org/ro-crate/specification/1.2/introduction.html","type":"aligned","note":"Optional evidence packaging profile; selected 1.2 must be pinned and mapped before use."},{"target":"WM-AI-003 AI Model Evaluation","type":"neighbor","note":"Registry parent is a conceptual specialization lead. The study adds controlled intervention contrasts; no unverified runtime inheritance or containment is asserted."},{"target":"WM-AI-004 AI Inference / Agent Run and WM-AI-006 Model Training / Fine-tuning Run","type":"neighbor","note":"Execution attempts have separate masters. A study can exist before runs and may reference optional training runs when retraining is part of its intervention."},{"target":"WM-AI-009 Evaluation Dataset / Benchmark and WM-AI-014 Ground-truth Label Set","type":"neighbor","note":"Frozen case and label references are used; dataset curation and label lifecycle are not owned here."},{"target":"WM-AI-011 Evaluation Harness and Scorer and WM-AI-012 Benchmark Result","type":"neighbor","note":"Scorer definitions and result records remain external. The study owns the rationale for comparing arms, not the master scores or implementation."},{"target":"Reusable experiment design and factorial experiment","type":"neighbor","note":"This root is an instantiated study record, not a design pattern. Multi-arm contrasts are allowed with declared settings; a single ablation cannot isolate all interactions."},{"target":"WM-AI-003","type":"parent"}],"interaction":{"identity":{"applicability":"required","items":["Authoritative master-system identifier in its namespace.","Stable owner-scoped identifier or UUID when no master identifier exists.","Content hash for integrity and deduplication, not a substitute for logical identity."]},"properties":{"applicability":"not-applicable","items":[]},"recognition":{"applicability":"optional","items":[]},"capabilities":{"applicability":"required","items":["Register study intent: Proposed, unimplemented local operation: register study intent.","Freeze comparison protocol: Proposed, unimplemented local operation: freeze comparison protocol.","Link execution evidence: Proposed, unimplemented local operation: link execution evidence.","Check contrast comparability: Proposed, unimplemented local operation: check contrast comparability.","Record analysis and interpretation: Proposed, unimplemented local operation: record analysis and interpretation.","Record review and release decision: Proposed, unimplemented local operation: record review and release decision."]},"hazards":{"applicability":"optional","items":[]},"interfaces":{"applicability":"required","items":["PROV-O: The PROV Ontology"]},"context":{"applicability":"required","items":["No jurisdiction or sector is assumed. Adopters must supply applicable evidence rights, privacy, ethics and retention rules.","English conceptual draft; no localization or legal compliance claim."]}},"sources":[{"title":"One variable at a time","url":"https://www.itl.nist.gov/div898/handbook/pri/section2/pri212.htm","note":"NIST"},{"title":"Two-Sample t-Test for Equal Means","url":"https://www.itl.nist.gov/div898/handbook/eda/section3/eda353.htm","note":"NIST"},{"title":"How do you select an experimental design?","url":"https://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm","note":"NIST"},{"title":"Randomized block designs","url":"https://www.itl.nist.gov/div898/handbook/pri/section3/pri332.htm","note":"NIST"},{"title":"PROV-O: The PROV Ontology","url":"https://www.w3.org/TR/prov-o/","note":"W3C"},{"title":"RO-Crate 1.2 Introduction","url":"https://www.researchobject.org/ro-crate/specification/1.2/introduction.html","note":"RO-Crate community"},{"title":"Statistical Comparisons of Classifiers over Multiple Data Sets","url":"https://www.jmlr.org/papers/v7/demsar06a.html","note":"Janez Demsar, research author"},{"title":"NeurIPS Paper Checklist Guidelines","url":"https://neurips.cc/public/guides/PaperChecklist","note":"NeurIPS research community"}],"openQuestions":["Complete direct source checks and selected-version verification, review the full statistical paper if a profile needs its methods, and restore independent external review before canonical promotion.","Develop nested instance schemas and fixtures for scorer drift, unmatched pairs, failed retries, label revisions, budget imbalance and outcome-informed protocol amendments.","Validate metric-specific analysis profiles with dependent observations, stochastic scorers, multiple contrasts and non-estimable uncertainty, and pin external model contracts and evidence export mappings.","Independent external provider review is absent.","Direct HTTP status and immutable source versions remain unverified.","Nested instance schemas, profile-specific validators and acceptance fixtures remain unimplemented.","Statistical methods for particular metrics, clusters, adaptive designs and resource-constrained studies need qualified selection.","Neighbor model contracts and machine-executable mappings remain unpinned."],"resources":{"spec":"/models/wm-ai-013-ablation-study/spec.yaml","agents":"/models/wm-ai-013-ablation-study/AGENTS.md","source":"https://github.com/ver-cy/world-models/tree/feat/mega-model-registry/research/runs/wm-ai-013"},"provenance":{"origin":"world-models research","builtFrom":["models/wm-ai-013-ablation-study/spec.yaml"],"providers":["Codex"],"researchStatus":"reviewable-draft","generatedAt":"2026-10-21T17:40:37Z","builder":"tools/build_cards.py@1.0.0"},"completeness":{"sections":{"classifiers":"filled","whatItIs":"filled","purpose":"filled","distinguishingFeatures":"derived","structure":"filled","agentConduct":"derived","ethics":"derived","owners":"filled","relations":"filled","interaction.identity":"filled","interaction.properties":"not-applicable","interaction.recognition":"missing","interaction.capabilities":"filled","interaction.hazards":"missing","interaction.interfaces":"derived","interaction.context":"filled","sources":"filled"},"notes":{"distinguishingFeatures":"Derived from boundary notes against neighbouring models.","agentConduct":"Derived from functions, policies, CRUD and access rules; prohibitions were not authored for agents as such.","ethics":"Sentences mentioning harm, privacy, consent or similar, collected from the specification.","interaction.properties":"Institutional or informational subject: no invented physical properties."},"score":0.75}}