objectivity
Enable an AI agent to assess whether a claim, assessment or decision is adequately constrained by evidence and explicit criteria rather than by the assessor's preferences, interests or identity.
Research draft, second pass
A second pass drafted this model: the structure a model of this thing needs, and what is known about it in the world. The line under this one says how the second half was obtained - researched against sources, or recalled without web access, in which case nothing here was read anywhere and every claim is a lead to verify. Unreviewed either way.
Researched by: Codex + Grok
Purpose and description
Enable an AI agent to assess whether a claim, assessment or decision is adequately constrained by evidence and explicit criteria rather than by the assessor's preferences, interests or identity.
Objectivity is the extent to which a representation, measurement, judgment, or procedure is constrained by the features of its object and by publicly shareable methods, rather than by the particular standpoint, interests, or idiosyncrasies of the agent who produces it.
It can be Define the bearer, intended use and applicable standard before assessing objectivity.; Trace plausible routes by which interests or discretion could influence evidence, interpretation or judgment.; Compare treatment of supporting and opposing evidence under the same relevance and quality criteria.; Request blinded reassessment, independent scrutiny or sensitivity analysis targeted at a specific concern.; Record a bounded objectivity judgment and attach unresolved challenges to downstream uses.; Recommend proceeding, qualifying a conclusion, obtaining further review or withholding reliance according to the assessment and decision stakes..
Distinguishing features
A correct result does not establish objectivity: inspect whether selective evidence or assessor interests could have produced it.
Agreement does not establish objectivity: check whether assessors share sources, assumptions or incentives that could explain their convergence.
Giving opposing positions equal weight does not establish objectivity: inspect whether their treatment follows consistent standards of evidential support.
Repeatability does not establish objectivity: a procedure may reproduce the same result while preserving an unexamined discretionary choice.
An objectivity assessment must identify which influences are inappropriate for the stated task; the mere presence of interpretation or values is insufficient to determine failure.
Scope
+ The claim, assessment, procedure or decision whose objectivity is being evaluated
+ The applicable sense of objectivity and the context in which it is required
+ Dependence on assessor identity, preferences, interests and discretionary choices
+ Evidence selection, counterevidence handling and consistency of evaluative criteria
+ Independent scrutiny, sensitivity tests and limits on objectivity judgments
- Truth or factual accuracy of the assessed claim itself
- Reliability and calibration of measurement instruments as standalone properties
- Fairness and justice of outcomes across affected groups
- Neutrality between competing parties or positions
- Ontological questions about whether an entity exists independently of minds
- General governance of evidence, datasets and research workflows
Characteristics
- Objectivity bearer
- Link to the specific claim, assessment, procedure or decision under evaluation Prevents evidence about a procedure from automatically becoming a judgment about every result or participant.
- Assessment standard
- Evidence responsiveness; procedural impartiality; assessor independence; cross-perspective robustness; explicitly defined combination Makes the intended meaning of objectivity explicit instead of treating different standards as interchangeable.
- Potentially compromising dependence
- Named assessor, incentive, preference or discretionary choice linked to a plausible influence on the result Directs scrutiny toward identifiable routes of influence without treating their mere presence as proof of distortion.
- Assessor substitution sensitivity
- Change in outcome units or categorical disagreement under documented assessor substitutions; untested where unavailable Shows whether relevantly equivalent assessors reach materially different results and supports investigation of why.
- Discretion sensitivity
- Outcome range, ranking changes or decision reversals across specified defensible analytical choices Reveals whether a conclusion depends on one contestable choice among reasonable alternatives.
- Assessment status
- Unassessed; insufficient evidence; supported within stated bounds; contested; compromised under the stated standard Allows bounded judgments and revision without implying a universal numerical objectivity score.
Where this came from
wikidata · CC0 1.0
Drafted structure
Bundle to layer to finding to question, as the second pass will find it: 5 bundles · 9 layers · 16 findings · 24 questions.
Objectivity standard Defines what must be independent of what for this particular assessment.
An agent cannot assess objectivity without identifying its bearer and the influences the task regards as inappropriate.
Bearer and use
Locates the objectivity claim and the use for which it matters.
Bounded objectivity claim
Record the exact result or procedure being assessed and avoid extending that judgment to an entire person or institution.
- Is objectivity being claimed for a particular conclusion, a decision, an assessment procedure or some combination? definition
- For which intended uses and circumstances must this objectivity judgment hold? boundary
Permitted and compromising influences
Separates legitimate task commitments from influences that would undermine the stated standard.
Explicit independence requirement
Record which preferences, interests or identity attributes should not determine the outcome, alongside relevant expertise and declared evaluative commitments.
- Which assessor attributes or interests should leave the outcome unchanged when relevant evidence and competence are held equivalent? definition
- Which goals, value judgments or contextual commitments legitimately shape this task, and who established that boundary? boundary
Evidence discipline Examines whether evidence constrains the conclusion consistently, including when it challenges a preferred result.
An assessment needs to distinguish evidence-responsive judgment from selective support for a predetermined conclusion.
Selection and weighting
Examines how material becomes eligible evidence and receives weight.
Symmetric evidence criteria
Record inclusion, exclusion and weighting decisions, including whether their justification changes with the direction of the evidence.
- What relevance and quality rules governed evidence selection, and were they specified before the preferred conclusion was known? provenance
- Would the same evidence receive the same weight if it supported the opposing conclusion? measurement
Counterevidence and revision
Tests whether contrary evidence can meaningfully change the assessment.
Demonstrable revisability
Record significant counterevidence, its treatment and the conditions under which the conclusion would be revised.
- What is the strongest identified counterevidence, and how did its assessment affect the conclusion or confidence? provenance
- What additional observation or successful criticism would require the agent to revise or withdraw the conclusion? action
Assessor and discretion dependence Investigates whether interests, identities or contestable choices materially control the outcome.
Objectivity concerns require evidence about routes of influence and their effects, beyond disclosure of possible bias.
Interest influence
Connects relevant interests to opportunities for influencing the assessment.
Influence path and safeguard
Record plausible influence paths, safeguards such as blinding or recusal, and evidence about whether those safeguards operated.
- Which interests or outcome preferences could affect selection, interpretation or approval, and through what specific discretionary power? provenance
- Which safeguard addresses each influence path, and what evidence shows that it operated in this assessment? measurement
Substitution and choice tests
Examines robustness to changes that should be irrelevant or are defensible alternatives.
Material dependence test
Record outcome changes under assessor substitution, identity masking or alternative defensible methods, and investigate their explanations.
- When relevant evidence and competence are held equivalent, how does the outcome change across assessors or masked identity conditions? measurement
- Which defensible changes to framing, weighting or analysis reverse the conclusion, and what explains those reversals? measurement
Scrutiny and warranted use Connects independent challenge to a bounded judgment and appropriate downstream reliance.
An agent must distinguish an asserted objectivity claim from one that has survived relevant scrutiny and know what unresolved concerns permit.
Independent challenge
Examines whether scrutiny can expose shared assumptions and contest the reasoning.
Effective critical access
Record what reviewers could inspect, their relevant independence and the disposition of substantive objections.
- Could reviewers inspect the evidence, exclusions and reasoning needed to challenge the objectivity claim? boundary
- Which reviewer dependencies or shared assumptions limit the independence of apparent agreement, and which objections remain unresolved? provenance
Bounded verdict and response
States the assessment result, its limits and the action it supports.
Qualified objectivity judgment
Record the supported status under the declared standard, distinguish missing evidence from demonstrated compromise, and specify revision triggers.
- Which parts of the objectivity claim are supported, untested or challenged, and does any demonstrated dependence compromise the stated standard? boundary
- Given the intended use and stakes, should the agent proceed, qualify reliance, seek a targeted reassessment or withhold reliance, and what would change that action? action
Evidence and external alignment What the world already says about this thing, gathered so the model can be checked against it.
A model that cannot be lined up against existing standards, identifiers and practice cannot be adopted by anyone who already uses them.
Reported evidence
Findings from the breadth pass, kept separate from the structural claims.
Kinds and varieties
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- Ontological objectivity: mind-independence of the object itself
- Epistemic objectivity: a claim's warrant does not rest on who the particular knower is
- Procedural or mechanical objectivity: following rules or instruments that restrain interpretation
- Intersubjective or aperspectival objectivity: invariance of a result across competent observers
- Journalistic objectivity: separation of reported fact from advocacy or the reporter's commitments
- Metrological objectivity: a measurement result not depending on a particular operator or instrument within a stated method
- Judicial and administrative impartiality: even-handed application of a rule to parties
- Strong or standpoint objectivity: examining the values and social location that shape inquiry rather than pretending they are absent
- Which of these kinds and varieties hold for the sense of objectivity this model covers, and on what evidence? provenance
Identifiers and schemes
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- LCSH - Objectivity - Library of Congress Subject Headings; used for the philosophical, scientific, and journalistic concept, not a single numbered code.
- Which of these identifiers and schemes hold for the sense of objectivity this model covers, and on what evidence? provenance
Standards and regulation
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- ISO 5725 (ISO): accuracy, trueness, precision, repeatability and reproducibility of measurement methods
- JCGM 200 VIM (JCGM / BIPM, in cooperation with ISO, IEC, IUPAC, IUPAP, IFCC, ILAC, OIML): vocabulary for measurement claims
- ISO 19011 (ISO): auditor objectivity and objective evidence in management-system audits
- ISO/IEC 17025 (ISO/IEC): impartiality and independence of testing and calibration laboratories
- ISO 9000 family (ISO): 'objective evidence' as the basis for conformity decisions
- ICH E9 Statistical Principles for Clinical Trials (ICH): blinding, unbiased estimation, and pre-specification as objectivity controls in medicines regulation
- Society of Professional Journalists Code of Ethics (SPJ): independence and minimization of harm; U.S. journalism's operational substitute for a named 'objectivity' statute
- Judicial codes of conduct (national benches and, internationally, the Bangalore Principles of Judicial Conduct, Judicial Integrity Group / UNODC): impartiality, recusal, and appearance of bias
- Which of these standards and regulation hold for the sense of objectivity this model covers, and on what evidence? provenance
Real-world use
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- A calibration laboratory reports a value with a reproducibility figure so another lab using the same method can treat the result as not tied to one technician or bench.
- A management-system auditor records only what is supported by objective evidence (records, observations, interviews) and recuses where a conflict of interest exists.
- A clinical trial uses randomization and double-blinding so efficacy estimates are not driven by investigator or patient expectation.
- A newsroom separates news from opinion and requires sourcing independent of the reporter's preferred narrative.
- A judge or hearing officer recuses when prior dealings with a party would make the decision look interest-laden.
- Peer review and preregistration are used in research to constrain post-hoc storytelling about a dataset.
- Inter-rater protocols in medicine, content moderation, and social measurement check whether a classification survives a change of observer.
- Which of these real-world use hold for the sense of objectivity this model covers, and on what evidence? provenance
Typical measurements
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- Cohen's kappa (chance-corrected inter-rater agreement) - 0.40-0.80 for usable agreement; <0.20 poor; >0.80 often treated as strong (scale theoretically −1 to 1) - dimensionless
- Intraclass correlation coefficient (ICC) for continuous ratings across raters - 0.50-0.90 in applied rating studies; 0-1 in the usual two-way models - dimensionless
- Reproducibility standard deviation sR (ISO 5725), between-laboratory - method- and measurand-specific; often a few percent of the mean in well-standardized chemical methods, much larger for complex social or sensory ratings - same as the measurand
- Measurement bias (estimate minus accepted reference value) - method-specific; laboratories usually require bias inside a stated maximum permissible error - same as the measurand
- Which of these typical measurements hold for the sense of objectivity this model covers, and on what evidence? provenance
Failure modes and hazards
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- Procedural compliance launders an interested outcome: the method was followed, so the result is labelled objective while the method itself encodes a party's frame.
- Observer-expectancy, confirmation, and selection effects persist under the label of objectivity.
- Journalistic both-sides-ism treats balance between parties as a proxy for constraint by the object, producing false equivalence.
- The view-from-nowhere pretence excludes relevant standpoint knowledge and hides who set the questions and instruments.
- A precise, highly repeatable method is mistaken for an unbiased one (precision without trueness).
- Conflicts of interest in audit, science, or appraisal are managed as paperwork rather than as threats to independence.
- 'Objectivity' is used rhetorically to dismiss testimony from affected groups without testing it against the object.
- Which of these failure modes and hazards hold for the sense of objectivity this model covers, and on what evidence? provenance
Regional variation
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- U.S. professional journalism long treated 'objectivity' as a newsroom doctrine; much of continental European journalism organizes instead around named political or civic editorial lines with separate fact-checking norms.
- German-language philosophy and science use Objektivität with a Kantian and later logical-empiricist history that does not map one-to-one onto English 'objectivity' in journalism.
- Standpoint and decolonial scholarship (widely taught in parts of Latin America, South Africa, and anglophone gender studies) treat classical value-free objectivity as a local, not universal, ideal.
- Metrology practice under ISO/IEC 17025 is comparatively uniform internationally; the philosophical and journalistic senses are not.
- State media systems that require 'correct guidance of public opinion' (notably in the PRC) reject Western news-objectivity doctrine while still using 'objective' for scientific and statistical reporting.
- Which of these regional variation hold for the sense of objectivity this model covers, and on what evidence? provenance
Neighbouring kinds and how to tell them apart
Reported by the breadth pass; each item needs checking against its source before it becomes normative.
- Neutrality - Neutrality is refusal to take a side; a method can be objective while supporting a definite factual conclusion. Test: does the procedure still constrain the result if the conclusion is politically inconvenient?
- Impartiality - Impartiality is even-handed treatment of parties; objectivity is constraint by the object. Test: a referee can be impartial between two false claims; an objective procedure can prefer one party if the evidence does.
- Intersubjectivity - Shared agreement among subjects can be shared bias. Test: would a differently situated but competent observer, or an independent method, still get the same result?
- Accuracy / trueness - Trueness is closeness to a reference value; a method can be operator-independent (objective in the metrological sense) and still biased. Test: compare to an independent reference, not only to repeatability.
- Independence (audit or editorial) - Independence is a structural relation to funders and subjects; it is a typical enabling condition, not the property of the claim. Test: an independent actor can still use a standpoint-laden method.
- Fairness - Fairness is a justice standard for how people are treated; objectivity is an epistemic or measurement standard. Test: a fair lottery is not an objective estimate of merit.
- Metaphysical realism - Realism is a thesis that some things exist mind-independently; objectivity is a property of representations and methods. Test: one can use objective methods on constructed objects (prices, diagnoses, legal statuses) without settling realism about them.
- Which of these neighbouring kinds and how to tell them apart hold for the sense of objectivity this model covers, and on what evidence? provenance
Sources
- Objectivity - Lorraine Daston and Peter Galison, Zone Books, 2007 - Historical kinds of scientific objectivity, especially mechanical objectivity versus trained judgment, and why 'letting the instrument speak' is a practice rather than a default.
- The View from Nowhere - Thomas Nagel, Oxford University Press, 1986 - The philosophical ideal of detaching from a particular perspective, and the limits of a complete view-from-nowhere.
- The Irreducible Complexity of Objectivity - Heather Douglas, Synthese 138(3), Springer, 2004 - A typology of objectivity (manipulable, detached, value-free, procedural, concordant, and so on) used in philosophy of science and policy.
- Whose Science? Whose Knowledge? Thinking from Women's Lives - Sandra Harding, Cornell University Press, 1991 - The 'strong objectivity' challenge: standpoint and social location as resources for, not mere contaminants of, objective inquiry.
- ISO 5725-1:1994 Accuracy (trueness and precision) of measurement methods and results - Part 1: General principles and definitions - International Organization for Standardization - How laboratories operationalize independence of results from particular runs, operators, and laboratories via trueness, precision, repeatability, and reproducibility.
- JCGM 200:2012 International vocabulary of metrology - Basic and general concepts and associated terms (VIM), 3rd edition - Joint Committee for Guides in Metrology / BIPM - Definitions of measurement, measurand, influence quantity, and related terms against which 'objective' measurement claims are judged in metrology.
- ISO 19011:2018 Guidelines for auditing management systems - International Organization for Standardization - Objectivity as an auditor principle: findings based on objective evidence, not on the auditor's interests or prior relationship with the auditee.
- ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories - ISO and IEC - Laboratory impartiality and structurally managed independence as the quality-system counterpart of measurement objectivity.
What the second pass must settle
- Does the registry intend objectivity to cover epistemic assessments, evaluative decisions and procedures together, or a narrower quality within XCT.QLT?
- Which neighbouring registered concepts own impartiality, neutrality, bias, reliability and truth, and which relations should connect them to this model?
- What task-specific standards can distinguish legitimate value commitments from influences that compromise objectivity?
- What evidence and thresholds are sufficient to move an assessment from insufficient evidence to supported or compromised in different domains?
- How should the model represent cases where assessor independence, evidence responsiveness and cross-perspective robustness support conflicting judgments?