Quality / Confidence

Identity of the quality assertion as a distinct resource

The quality/confidence record is a resource in its own right, distinct from the subject, from the measure and from the assessing activity. It must be independently identifiable so that it can be cited, superseded, disputed, embargoed and retained on its own schedule. DQV expresses this as a typed resource (QualityMeasurement, QualityAnnotation, QualityCertificate, UserQualityFeedback, QualityPolicy) that is linked to, rather than embedded in, the assessed resource; PROV then attaches generation and attribution to that resource.

1. Which identifier identifies this quality assertion itself, as distinct from the subject it describes and the measure it applies? identity
2. Is this record a computed measurement, a human quality annotation, a certificate, a policy statement, or user feedback? classification
3. What natural key determines whether two records are the same assertion rather than two independent assessments of the same subject? identity
4. Who is authorised to mint and retire identifiers for quality assertions within the adopting Dimension? authority

Subject attachment and assessment scope

An assertion is meaningless unless the assessed unit is explicit: the whole resource, one distribution, a subset, a feature type, a single instance, one attribute, or a population represented by a sample. ISO 19157-1 makes scope an explicit component of a data quality evaluation, and DQV's computedOn names the exact resource assessed. Ambiguous scope is the most common way an honest score becomes a misleading one.

1. What exactly was assessed: the whole resource, a distribution, a subset, a field, or a single instance? composition
2. Does the assertion apply only within a restricted extent, population segment, or operating condition, and how is that restriction expressed? constraint
3. Was the whole scope inspected or was a sample used to infer a property of a population? measurement
4. May one assertion cover several subjects at once, and if so how is per-subject traceability preserved? relationship

Dimension and category vocabulary binding

DQV separates Dimension (a quality-related characteristic that matters to consumers) from Metric (an abstract procedure for assessing it) and groups dimensions into Categories, but deliberately declines to prescribe one authoritative list. Multiple valid vocabularies exist in parallel — ISO/IEC 25012's fifteen characteristics under inherent and system-dependent viewpoints, ISO/IEC 5259-2 for analytics and ML data, ISO 19157-1's geographic quality components, and the European Statistics Code of Practice output principles. The mixin therefore binds a vocabulary per subject class rather than embedding one, and records that binding as a governed profile.

1. Which dimension vocabulary is bound for this subject class, at which version, and by whose authority? interoperability
2. Is the dimension inherent to the data itself or dependent on the surrounding system and context of use? classification
3. Which dimensions are mandatory for this subject class before it may be published or relied upon, and which are optional? requirement
4. Where two bound vocabularies define overlapping or contradictory dimensions, which one governs? exception

Measure definition and registration

A score is uninterpretable unless the measure that produced it is itself a registered, versioned, dereferenceable object with a definition, parameters, a value type and a value structure. ISO 19157-1 specifies the components and content structure of data quality measures; ISO/TC 211 publishes these as normative machine-readable schemas; ISO/FDIS 19157-3 specifies a register of measures maintained under ISO 19135-1 registration procedures. That register standard is still at FDIS stage, so registration governance is adopted here as a defensible pattern rather than a settled normative requirement.

1. Which registered measure produced this value, at which version, and where is its definition dereferenceable? identity
2. What parameters, tolerances or thresholds were bound when the measure was applied, and what are their values? constraint
3. What value type and value structure does the measure emit — a single value, a ratio, a count, a matrix, or a coverage? measurement
4. How is a measure superseded or retired, and what happens to results already produced by the retired version? lifecycle

Domain-specific dimension extension

Additional dimensions (for example ML-specific characteristics in ISO/IEC 5259-2, linked-data interlinking, or community uniqueness) are first-class local concepts with provenance of who added them. The full list of extra ISO/IEC 5259-2 characteristics is not fully extractable from freely published text and remains a documented gap.

1. What identifier and definition distinguish this extension dimension from standard catalogue codes? identity
2. Who authorised the extension and against which standard extension clause? authority
3. If an ISO/IEC 5259-2 additional ML characteristic is claimed, is the official characteristic name cited from the standard text rather than inferred? evidence

Evaluation procedure, sampling and environment

The executed procedure must be recorded with enough fidelity to be repeated: full inspection versus sampling, the sampling design, the data splits used, the executing agent (human, automated or hybrid), the tool and its version, and the evaluation environment. ISO/IEC TS 4213 makes evaluation setup, preprocessing, training/test/validation splits, cross-validation, documented algorithm and hyperparameters, environment and baselines explicit control criteria; ISO 19157-1 makes the evaluation method a reported component.

1. Which documented procedure and which tool version executed this assessment? process
2. If sampling was used, what was the sampling design, the achieved sample size, and the resulting representativeness claim? measurement
3. Was the assessment automated, human-judged, or hybrid, and where did human judgement enter? provenance
4. What preprocessing, exclusion or filtering was applied before measurement, and could it bias the result? quality

Reference basis, ground truth and traceability

Every quality claim is a comparison against something. That reference must be named: a specification, a reference dataset, an adjudicated ground truth, a calibrated standard, or an expert panel. VIM defines accuracy, trueness and precision strictly relative to a true or reference value, and defines metrological traceability as the property that relates a result to a reference through a documented unbroken chain of calibrations. Where no defensible reference exists, the assertion must say so rather than imply one.

1. Against which reference, specification or ground truth was the subject compared, and who adjudicated it? evidence
2. Is there an unbroken documented chain relating this result to a stated reference, and where is it recorded? provenance
3. What is known about the quality and error rate of the reference itself? quality
4. If no independent reference exists, how is that stated so the result is not read as accuracy? exception

Result value, scale and value structure

The result carries a value plus everything needed to interpret it: expected data type, scale type (nominal, ordinal, interval, ratio), value structure (single value, ratio, count, matrix, coverage) and, where the value is a quantity, a dereferenceable unit reference. DQV recommends dereferenceable unit identifiers rather than unit strings. Ordinal grades must be marked as ordinal so that consumers do not average them.

1. What is the result value and what data type and value structure does it use? measurement
2. What scale type applies, and which arithmetic operations are therefore invalid on this value? constraint
3. If the value is a quantity, which dereferenceable unit identifier applies? interoperability
4. What is the permitted value domain, and how are null, not-applicable and not-assessed distinguished from a poor score? validation

Uncertainty and error model

A value without an uncertainty statement cannot support a defensible conformity decision. JCGM GUM defines the evaluation of measurement uncertainty and its supplements cover distribution propagation and multiple output quantities; VIM defines standard, combined and expanded uncertainty, coverage interval, coverage probability and coverage factor. This model requires that any quantitative result declare either an uncertainty statement or an explicit, reasoned statement that none was evaluated. Uncertainty about the value is distinct from confidence in the assertion, and the two must not be merged.

1. What uncertainty is attached to the result, expressed as a standard uncertainty, an expanded uncertainty, or an interval? measurement
2. How was the uncertainty evaluated, and which components dominate the budget? process
3. If no uncertainty was evaluated, why not, and what interpretation is thereby forbidden? exception
4. How is the uncertainty of the value kept distinct from the confidence in the assertion as a whole? definition

Object and semantics of a confidence statement

Confidence in this model is a statement about the assertion — how much the assessment can be relied upon — not a property of the subject and not a probability that a proposition about the world is true. GRADE frames certainty as certainty that the true value lies on one side of a threshold or within a range, which is explicitly about the evidence base rather than about the estimate alone. Conflating confidence with a model's output probability, with coverage probability, or with a risk score is the principal failure mode this finding exists to prevent.

1. What is this confidence statement about: the assertion, the underlying value, the method, or the subject? definition
2. Is the confidence value a probability, and if not, what is it? classification
3. What action is an agent permitted or forbidden to take at each confidence level? decision
4. Where the subject is a model output that carries its own score, is that score treated as confidence, and is it calibrated? quality

Confidence scale declaration and calibration

A confidence value is uninterpretable without its scale. The scale must be named, versioned and dereferenceable, with its ordered terms and any numeric mapping stated. GRADE's four-level certainty scale is one such governed scale; a model's softmax output is not. Where a numeric confidence is claimed to be calibrated, the calibration evidence — reference set, calibration method, date and observed calibration error — must be recorded, otherwise the value is reported as uncalibrated.

1. Which confidence scale is used, at which version, and where are its ordered terms defined? interoperability
2. Is there an authorised numeric mapping for the scale terms, and is cross-scale comparison permitted? constraint
3. What evidence supports a claim that numeric confidence values are calibrated, and when was it last established? evidence
4. How is an uncalibrated confidence value marked so downstream agents do not treat it as a probability? quality

Confidence basis: evidence and agreement

Confidence must be decomposed into named reasons rather than asserted. GRADE names domains that lower certainty (risk of bias, imprecision, inconsistency, indirectness, publication bias) and domains that raise it (large effect, dose-response gradient, plausible confounding that would only strengthen the finding). This model generalises that pattern: record the type, amount and consistency of evidence, the degree of agreement among independent assessments, and each rating-down or rating-up decision with its reason, so the grade is reconstructible and challengeable.

1. What type and amount of evidence underpins this assertion, and how consistent is it? evidence
2. How much do independent assessors, methods or sources agree, and how was agreement measured? quality
3. Which factors lowered or raised the confidence rating, and what is the recorded reason for each adjustment? decision
4. What evidence that would materially change the rating is known to be missing or inaccessible? exception

Metaquality

A quality report can itself be wrong, biased or unrepresentative. Metaquality elements record confidence in the quality evaluation, whether the evaluated sample represents the quality unit, and whether quality is homogeneous across the unit. This is quality-of-quality, not a second copy of the host score.

1. What metaquality confidence is assigned to this quality result, and on what evidence? quality
2. How representative of the declared quality unit is the evaluated sample or census? evidence
3. Is quality homogeneous across the unit, or do partitions differ enough that a single score is misleading? quality
4. Who assessed metaquality, and is that party independent of the original assessor? ownership

Evidence artifacts, integrity and re-execution

An assertion is only as defensible as the material a challenger can retrieve. W3C VC Data Model 2.0 provides an evidence property, cryptographic securing mechanisms and credential status for tamper-evidence and revocation; the EU AI Act requires technical documentation and retained logs that substantiate declared performance. This finding requires that each assertion either point to retrievable, integrity-protected evidence with re-execution instructions, or declare explicitly that it is an unsubstantiated judgement.

1. Which retrievable evidence items substantiate this assertion, and what does each one show? evidence
2. How is the integrity of each evidence item protected and verified at retrieval time? security
3. What is needed to re-execute this assessment and reproduce the result within stated tolerance? process
4. If no evidence is retained, is the assertion explicitly marked as an unsubstantiated judgement? quality
5. Is any evidence item access-restricted, and what may a consumer be told when they cannot retrieve it? access

Quality requirement and acceptance threshold

A threshold is a governed object separate from both the measure and the result: it names the measure, the acceptance limit, the direction of the comparison, its owner and its effective period. Because measurement uncertainty affects conformity decisions, the decision rule must state how uncertainty is handled — for example acceptance limits with guard bands — rather than comparing a point estimate to a limit and calling it conformity.

1. What acceptance limit applies to this measure, in which direction, and who set it? requirement
2. How does the decision rule account for measurement uncertainty when comparing the result to the limit? decision
3. On what basis was the threshold chosen — regulation, contract, product specification, or internal convention? authority
4. What happens to existing verdicts when a threshold changes? lifecycle

Conformance verdict and severity

A verdict is a derived assertion that cites both a result and a requirement. Constraint validation supplies a genuinely binary conformance signal with organisational severities (violation, warning, informational); graded measurement supplies a score. The model keeps them distinct: a verdict must name the requirement it was evaluated against, and a score may never be presented as a verdict. Severity classifies consequence, not degree of failure.

1. What is the verdict, against which requirement, and using which result? validation
2. What severity is attached to each non-conformance, and what does that severity oblige the consumer to do? constraint
3. Is the verdict reproducible from the recorded result, requirement and decision rule alone? provenance
4. How are partial, conditional or waived conformance states represented without collapsing them into pass? exception

Intended use, fitness for purpose and declared limitations

Quality is only decidable relative to a purpose: the European Statistics Code of Practice defines relevance by user need, ISO 19157-1 frames quality information as helping users decide whether data suffice for their particular application, and the EU AI Act requires providers to declare accuracy levels and metrics in the instructions for use. A fitness statement must therefore name the purpose it was judged against and, symmetrically, state the known defects, exclusions and uses the assertion does not support — otherwise absence of a caveat is read as absence of a limitation.

1. For which stated purpose was fitness judged, and by whom? decision
2. Is the subject judged fit, fit with conditions, or unfit for that purpose, and on which assertions does the judgement rest? validation
3. What known defects, exclusions or systematic errors are present that a consumer must be told about? quality
4. Which uses does this assessment explicitly not support, and where is that recorded so it travels with the data? constraint
5. What happens to the fitness judgement when the intended purpose changes? lifecycle

Assessment time versus subject state time

At least three times must be distinguishable: the time of the subject state that was assessed, the time the assessment produced its result, and the time the assertion was recorded or ingested. SOSA's separation of phenomenonTime from resultTime is the canonical pattern. All times use RFC 3339 with seconds and an explicit offset or Z; a date alone is never sufficient and is never an identifier.

1. Which state of the subject, at which instant or interval, does this assertion describe? temporal
2. When was the assessment executed and when did its result become available? temporal
3. When was the assertion recorded or ingested into the adopting Dimension, and how does that differ from the result time? provenance
4. What time precision and clock source are used, and is the precision adequate for the decisions taken on this assertion? quality

Validity period, currency and re-assessment

Assertions have a shelf life. Credential models express this with validFrom/validUntil; official statistics treat timeliness and punctuality as governed dimensions. This model requires an explicit validity policy per measure: a validity window or a decay statement, a re-assessment cadence, and defined consumer behaviour when an assertion is stale — never silent reuse of an expired score.

1. From when until when may this assertion be relied upon? temporal
2. What must a consuming agent do when the assertion is past its validity window? state
3. How often must this measure be re-assessed, and what events force early re-assessment? process
4. Does quality on this dimension decay predictably over time, and is that decay modelled or merely assumed? measurement

Assertion lifecycle, supersession and immutability

Once issued, a quality assertion is a historical fact about what was believed at a time and must not be silently edited. Register practice models item status (valid, superseded, retired) with explicit supersession pointers; credential practice adds revocation through status checking. This model requires a declared state set, an append-only supersession chain, and a distinction between correcting a defective assertion (retract plus reissue with reason) and producing a new assessment (new assertion, prior one remains valid for its period).

1. What are the permitted lifecycle states of an assertion and which transitions are legal? lifecycle
2. Is this record a correction of a defective assertion or an independent new assessment? state
3. What guarantees that an issued assertion is not altered after the fact, and how is that verifiable? security
4. When an assertion is retracted, what must happen to decisions and derived assertions that relied on it? event

Assessor attribution, competence, authority and independence

A quality assertion inherits credibility from its author. PROV supplies attribution and qualified attribution with roles; DQV distinguishes producer measurements, third-party certificates and user feedback precisely because their standing differs. This model additionally records competence or accreditation where claimed, the authority under which the assertion is issued, and independence or conflict of interest relative to the subject's owner — a self-assessment and an independent audit are not interchangeable and must not be presented as equivalent.

1. Which agent produced this assertion, in which role, and on whose behalf? ownership
2. Under what authority, accreditation or delegated mandate is this assertion issued? authority
3. What is the assessor's relationship to the subject's owner, and is any conflict of interest declared? provenance
4. What evidence supports the assessor's competence for this specific measure? evidence

Provenance lineage and derived assertions

Assertions are frequently derived: rolled up across parts, weighted into composite indices, propagated to downstream products, or recomputed from other assertions. PROV's wasDerivedFrom, used and wasGeneratedBy make that lineage explicit, and DQV's quality-metadata container lets certificates, policies, measurements and annotations carry joint provenance as a group. Aggregation is a lineage operation with its own risks: composite scores hide dimension-level failures and are invalid across incompatible scales, so the aggregation function, weights and their justification must be recorded with the derived assertion.

1. From which assertions, datasets or activities was this assertion derived? provenance
2. If this is a composite or rolled-up score, what function and weights produced it and who approved them? composition
3. Are the inputs on compatible scales and scopes such that aggregation is meaningful? constraint
4. Can a consumer recover the dimension-level results behind a composite score? interoperability
5. How does an assertion about an input propagate to products derived from that input? relationship

External alignment, mappings and recorded conflicts

External standards are alignments, not conformance claims. This model maps its slots to DQV classes and properties, to ISO/IEC 25012 and ISO/IEC 5259-2 dimension vocabularies, to ISO 19157-1 measure components, to JCGM uncertainty terms and to PROV attribution, recording for each mapping whether it is exact, broader, narrower or merely related. Conformance to any of these is asserted only where tested evidence exists; recorded conflicts include the fact that constraint-validation severities are not quality grades and that credential formats carry no confidence property.

1. To which external term does each slot of this model map, and at what mapping strength? interoperability
2. Is conformance to a cited standard claimed, and what test evidence supports the claim? evidence
3. Which known conflicts or semantic mismatches exist between aligned standards, and how are they resolved? exception
4. How are mappings maintained when an aligned standard is revised? lifecycle

Comparability and non-comparability rules

The most damaging misuse of quality data is comparing values that are not comparable. Coherence and comparability are treated as a governed output-quality principle in official statistics precisely because comparability must be established, not assumed. Two results are comparable only if the measure and its version, the parameter binding, the scope and population, the scale and unit, the reference basis and the validity period are compatible. Confidence grades on different scales are never comparable and never averageable.

1. Under what stated conditions may two results of this measure be compared or trended? constraint
2. Which combinations of scales, units or confidence vocabularies are explicitly forbidden from being combined? validation
3. How is a break in comparability caused by a method or threshold change signalled to consumers? event
4. When results are normalised for comparison, what normalisation was applied and is it reversible? process

Competing assessments, disagreement and rectification

Several parties may legitimately assess the same subject and disagree — DQV explicitly accommodates producer measurements, third-party certificates and user feedback side by side, since fitness for purpose varies by consumer. Disagreement must be represented, not silently resolved by last-writer-wins. Where assertions concern personal data, disagreement acquires legal force: the accuracy principle obliges rectification or erasure of inaccurate data, and data subjects hold rights to rectification and erasure, so a dispute channel with defined outcomes and deadlines is mandatory rather than optional.

1. Which other assertions cover the same subject, scope and dimension, and do they conflict? relationship
2. What precedence rule applies when assertions conflict, and who owns that rule? decision
3. How does an affected party challenge an assertion, and what outcomes and deadlines apply? exception
4. Where the assertion concerns personal data, what rectification or erasure duty applies and how is it evidenced? privacy
5. While a dispute is open, how is the assertion's disputed status made visible to consumers? state

Access sensitivity and disclosure of quality statements

A quality statement can be more sensitive than the subject it describes: a low accuracy score, an open dispute or an unresolved defect can be commercially or legally damaging, and evidence packages may contain personal or confidential data even when the subject does not. Access must therefore be classified independently at bundle, layer, finding and artifact level, with a defined redacted view for consumers who are entitled to know that a limitation exists but not to see the underlying evidence, and with a bias toward disclosing limitations that affect safety.

1. What access classification applies to this assertion, and does it differ from that of the subject? access
2. What may a consumer without full access be told, and what must never be inferable from the redacted view? security
3. Which limitations must be disclosed regardless of classification because they affect safety, rights or legal duties? requirement
4. Is the assertion under embargo before a scheduled release, and who may lift it? authority

Retention, deletion and legal hold of assertions and evidence

Retention of quality records is contested between opposing duties. Regulated AI providers must retain technical documentation, logs and performance evidence that substantiate declared accuracy; data protection law imposes storage limitation and grants erasure rights over personal data, including within evidence packages. This finding requires an explicit retention schedule per assertion class and per evidence artifact, a deletion mechanism that preserves the integrity of the historical record where the assertion itself must survive, and a legal-hold override that suspends scheduled deletion.

1. How long must this assertion and each of its evidence artifacts be retained, and on what basis? retention
2. When evidence must be deleted but the assertion must survive, what is retained and what is destroyed? process
3. How is an erasure request affecting quality evidence executed and evidenced without falsifying the audit trail? privacy
4. What suspends scheduled deletion, who may impose and release a hold, and how is the hold recorded? authority
5. How is a conflict between a mandatory retention duty and an erasure duty resolved and documented? exception