Data Schema / Data Contract

Contract and schema resource identity

A contract or schema must be identifiable independently of where its bytes are stored. ODCS requires a contract id plus apiVersion and kind; JSON Schema assigns a schema resource a canonical URI through $id and declares its dialect through $schema; a schema registry assigns a subject plus a numeric schema id; Avro derives a content fingerprint from Parsing Canonical Form. These are four different identity mechanisms with different guarantees, and an agent must know which one is authoritative in a given deployment.

1. Which identifier is the authoritative master-system identifier for this contract, and which system of record issues it? identity
2. Does the contract carry a governed global identifier or IRI in addition to its local identifier, and is that IRI resolvable? identity
3. Which specification and dialect does this artefact declare itself to be written in, and is that declaration machine-readable? definition
4. Is a content fingerprint computed for the schema, and over which canonical form is it computed? evidence

Governed subject and granularity

A contract binds to something concrete: a table, a topic subject, a file layout, a message payload or an API resource. The binding target and its granularity determine what a validation failure means. ODCS expresses this through domain, dataProduct, tenant and the physical name and type of each schema object; a schema registry expresses it through the subject and its naming strategy; DCAT expresses it as a conformance link from a dataset or distribution.

1. Which specific datasets, topics, tables or message types does this contract govern, and by what binding mechanism? relationship
2. At what granularity does one contract instance apply - one object, one product, one domain or one tenant? classification
3. Does the contract govern the entire target or only a declared subset of its fields and partitions? composition
4. What happens when two contracts claim authority over the same target? exception

Dialect, vocabulary and kind

The language that interprets the schema must be explicit: ODCS apiVersion and kind DataContract; JSON Schema $schema meta-schema URI and $vocabulary set; OpenAPI jsonSchemaDialect or OAS dialect id; SHACL Core versus SHACL-SPARQL. Companion JSON Schemas for ODCS and OAS are non-authoritative if they conflict with the standard text.

1. Is this artefact a DataContract, a JSON Schema document, a SHACL shapes graph, an Avro schema, or an OpenAPI Schema Object, and which kind value is declared? classification
2. Which meta-schema, apiVersion or dialect URI must a processor load before interpreting keywords? interoperability
3. Which vocabularies are required versus optional, and must a processor refuse the schema if an unknown required vocabulary is declared? requirement
4. If a companion JSON Schema disagrees with the ODCS or OpenAPI standard text, which source is authoritative? authority

Version designation and immutability

ODCS requires a version on every contract. Semantic Versioning supplies the widely used rules: MAJOR for incompatible change, MINOR for backward-compatible addition, PATCH for backward-compatible fixes, with the hard rule that a released version's contents MUST NOT be modified. A schema registry adds an orthogonal monotonic version number per subject plus a globally unique schema id. An agent must not conflate a contract's semantic version, a registry version number and a fingerprint - they answer different questions.

1. Which versioning scheme governs this contract, and what rule decides when the major component increments? constraint
2. Once published, is the version immutable, and what mechanism prevents in-place edits? constraint
3. Which version precedes and which supersedes this one, and how is that chain expressed to consumers? relationship
4. How many versions may be simultaneously in force, and how does a consumer pin to one? state

Contract status and lifecycle states

ODCS makes status a required field. Status is what tells a consumer whether the contract may be relied on, and it is distinct from version. Registry-style governance adds a registration status assigned by a registration authority, and DCAT-AP requires a controlled vocabulary for distribution status. The set of permitted states, the permitted transitions and who may perform each transition must be explicit; an unconstrained free-text status is a governance failure disguised as flexibility.

1. What is the closed set of permitted status values, and is it drawn from a controlled vocabulary? classification
2. Which status transitions are permitted, and which are forbidden or require a waiver? lifecycle
3. At what instant does a status change take effect, and is that instant recorded separately from when it was observed? temporal
4. Which statuses make the contract binding on producers, and which make it advisory only? authority

Ownership, stewardship and approval authority

The registry names a data product owner or data steward as maintainer. ODCS carries team and roles sections; the deprecated Data Contract Specification carries an owner in its info block. Ownership answers accountability; approval authority answers who can bind the organisation to a change. These are frequently the same person and must still be modelled separately, because delegation and vacancy are normal states.

1. Which single party is accountable for the correctness of this contract, and how is that party identified? ownership
2. Who is authorised to approve a breaking change, and is that authority delegable? authority
3. What is the fallback when the named owner is absent, has left, or the role is vacant? exception
4. Which agent is recorded as having authored each version, distinct from who approved it? provenance

Object and property structure

ODCS models a schema as an array of objects, each with a required name and a properties array; each property has a required name and optional logicalType, physicalType, required, unique, items for arrays and nested properties for objects. JSON Schema expresses the same shape through properties, items, prefixItems and applicator keywords; SHACL through node shapes carrying property shapes. Nesting depth, array element typing and the treatment of additional or unexpected properties are the decisions that most often break consumers.

1. What is the full set of objects and properties, including nesting depth and array element definitions? composition
2. Is the structure closed, or may an instance carry properties not declared in the contract? constraint
3. Does property ordering carry meaning for the physical representation, and where is that ordering authoritative? constraint
4. Which property definitions are reused from a shared definitions library rather than declared locally? composition

Keys, uniqueness and declared relationships

ODCS v3.1.0 added relationships (foreign keys), supporting property-level relationships with an implicit from field, schema-level relationships with explicit from and to, composite keys expressed as arrays, and nested property references using dot notation. It also carries primaryKey, primaryKeyPosition and unique on properties. Declared referential relationships are assertions about data that a schema validator alone cannot check - they need a referential validation step against a second dataset.

1. Which properties form the primary key, and in what order for a composite key? identity
2. Which uniqueness constraints hold beyond the primary key, and over what scope are they evaluated? constraint
3. Which declared relationships point at other datasets, and is referential integrity enforced or merely asserted? relationship
4. What is the agreed behaviour when a referenced target row is missing or arrives late? exception

Composition, references and shape targets

JSON Schema applicators apply subschemas in place or to children via $ref, $defs, allOf, anyOf, oneOf, not, if/then/else, properties, prefixItems and $dynamicRef. $id in a subschema starts a new resource. SHACL distinguishes node shapes from property shapes with sh:path (predicate, inverse, sequence, alternative, quantified paths) and targets class, node, subjects-of and objects-of. Avro references named types by fullname and uses unions instead of oneOf. OpenAPI Schema Object $ref follows JSON Schema 2020-12 inside schemas.

1. Which reusable definitions exist, and which $ref, named-type or shape IRIs must be resolved before evaluation? relationship
2. How are subschemas combined (allOf, anyOf, oneOf, not, union), and does adding a constraint follow open-world JSON Schema rules or Avro field lists? composition
3. Which nodes does each SHACL shape target, and what property path is constrained from the focus node? relationship
4. What happens if a reference cannot be dereferenced, or if $dynamicRef depends on evaluation-time dynamic scope? exception

Logical to physical type mapping

ODCS deliberately separates logicalType (string, date, timestamp, number, integer, object, array, boolean) from physicalType (the platform-specific type such as VARCHAR or INT) and physicalName. Avro layers logical types (decimal, uuid, date, time, timestamp, local-timestamp, duration) over primitive representations. The mapping is lossy in both directions: precision, signedness and nullability behave differently across engines, and the contract must state which side is authoritative.

1. What logical type is declared for each property, and from which type vocabulary is it drawn? classification
2. What physical type and physical name does each property take in each target platform? interoperability
3. For numeric and decimal properties, what precision, scale and rounding behaviour is guaranteed? measurement
4. How is absence distinguished from an explicit null and from a defined sentinel value? definition

Encoding, units and temporal conventions

Two datasets with identical structure can still be incompatible over character encoding, decimal separators, unit of measure and time representation. RFC 3339 requires seconds and an explicit offset, permits Z, treats -00:00 as an unknown local offset and allows second value 60 only for a leap second. Avro further distinguishes global-timeline timestamps from timezone-independent local timestamps. ODCS carries a unit attribute on quality rules; units on data properties themselves are a known weak point.

1. How are date and time values represented, and does the representation carry an explicit offset? temporal
2. Which timestamp property carries event time and which carries observation or ingestion time? temporal
3. What character encoding, collation and normalisation form apply to text properties? constraint
4. For quantity properties, what unit of measure is guaranteed and is it stated in the contract or assumed? measurement

Business term and value domain binding

ODCS provides authoritativeDefinitions at contract and property level, letting a property point at the definition that governs its meaning. The ISO/IEC 11179 family formalises this as a separation between the conceptual level (data element concept, conceptual domain) and the representational level (data element, value domain), administered by a registration authority. DCAT-AP shows the jurisdictional variant: certain properties must draw values from named controlled vocabularies. Without this binding, a schema is a set of labelled boxes.

1. Where is the authoritative definition of each business-significant property held, and is it dereferenceable? definition
2. Which properties draw values from an enumerated value domain or code list, and which list version applies? classification
3. Is the concept registered in a metadata registry as an administered item, and what is its registration status? provenance
4. Where does the contract's local meaning of a property deliberately diverge from the governed definition? exception

Structural and value constraints

JSON Schema validation keywords impose requirements on instances: type, required, enum, pattern, numeric bounds and length bounds are assertions. The format keyword is different: under the format-annotation vocabulary it is only an annotation, and only under the format-assertion vocabulary is it a checked assertion. SHACL offers the parallel set as constraint components on property shapes. An agent must record, per constraint, whether it is enforced, annotated or merely documented, because this determines whether a violation is detectable at all.

1. What value constraints apply to each property, expressed in which constraint vocabulary? constraint
2. For each declared constraint, is it an enforced assertion, an annotation, or documentation only? validation
3. Which properties are mandatory, and does mandatory mean present, non-null, or both? requirement
4. Which declared formats are actually validated by the deployed toolchain, and which are advisory? quality

Cross-field, conditional and cross-dataset constraints

The constraints that matter most operationally are rarely single-property. Conditional application (if one property has a value, another becomes required), mutual exclusivity, ordering between two timestamps, and referential checks against another dataset all fall outside what a single-property type declaration can express. JSON Schema handles some through applicator keywords, SHACL through shape-level constraint components, and ODCS pushes the remainder into SQL or custom quality rules. Recording where each constraint is expressed is essential, because it determines who can execute it.

1. Which constraints apply conditionally, and on what condition are they triggered? constraint
2. Which constraints require reading a second dataset, and who is responsible for executing them? process
3. In which language or engine is each non-structural constraint expressed, and is that expression portable? interoperability
4. At what point in the data flow is each constraint evaluated - on write, on publish, or after the fact? process

Quality rule declaration, threshold and consequence

ODCS defines four rule types - text (human-readable, not yet executable), library (predefined metrics such as nullValues, missingValues, invalidValues, duplicateValues, rowCount), sql (a query returning a numeric or boolean value) and custom (vendor-specific, for example Soda, Great Expectations, dbt or Monte Carlo). Rules carry a dimension drawn from accuracy, completeness, conformity, consistency, coverage, timeliness and uniqueness; a unit; a comparison operator from a set of eight; and governance attributes severity, businessImpact, scheduler and schedule. A threshold without a stated consequence is not an expectation, it is a dashboard.

1. What type is each quality rule, and is it executable as written or only human-readable? classification
2. What metric, comparison operator, threshold value and unit define pass or fail for each rule? measurement
3. What severity and business impact attach to a failure, and does failure block publication? requirement
4. On what schedule is each rule evaluated, and against which slice of the data? temporal
5. Which quality dimension does each rule serve, and is the dimension vocabulary controlled? classification

Validation outcome and conformance evidence

SHACL defines the validation report as the standard outcome, carrying conformance and per-violation detail. JSON Schema defines four structured output formats - flag, basic, detailed and verbose - with standardised location reporting and separation of errors from annotations. OpenLineage carries dataQualityAssertions and dataQualityMetrics as input dataset facets on run events. An agent needs to know which output format is produced, where it is kept, for how long, and whether it is admissible as evidence of compliance.

1. In what output format are validation results produced, and does it identify the failing instance location? evidence
2. What exactly does a conformance result assert, and against which contract version was it evaluated? validation
3. How long are validation reports retained, and what triggers their deletion? retention
4. Which timestamps are recorded for a validation run, distinguishing the data's event time from the run's observation time? temporal

Compatibility mode and breaking-change determination

A registry-governed compatibility mode is drawn from BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE and NONE, configurable globally or per subject. The mode is not cosmetic: under backward compatibility consumers must be upgraded first and permitted changes are field deletion and addition of optional fields with defaults; under forward compatibility producers must be upgraded first and permitted changes are field addition and deletion of optional fields with defaults. Transitive modes extend the check across all prior versions rather than only the immediate predecessor.

1. Which compatibility mode is in force for this contract, and is it set globally or per subject? constraint
2. Given the mode, which party must be upgraded first when a new version is published? process
3. What test determines that a proposed change is breaking, and is that test automated? decision
4. Is compatibility checked only against the immediately previous version or against all prior versions? constraint

Schema resolution, defaults and identifier stability

Avro specifies resolution between a writer's schema and a reader's schema: fields match by name, writer fields absent from the reader are ignored, reader fields absent from the writer are filled from defaults and error without one, aliases may map old names to new, and defined type promotions apply (int to long, float or double; long to float or double; float to double; string and bytes interchangeably). Iceberg takes a different route, tracking columns by identifier rather than name so a rename does not invalidate stored data, and evolving partition and sort order independently of schema. The contract must say which mechanism it relies on.

1. Which properties carry default values, and what do those defaults mean when a writer omits the field? definition
2. How are renames handled - through aliases, through stable column identifiers, or not at all? interoperability
3. Which type changes are treated as safe widening, and which force a new major version? constraint
4. Are partitioning and sort order versioned independently of the schema, and who tracks that? composition

Canonicalization, fingerprints and language alignments

Avro Parsing Canonical Form strips docs and aliases, expands fullnames, orders attributes and removes whitespace so two schemas that serialise identically in PCF are the same for readers. Fingerprints of PCF (SHA-256, MD5, CRC-64-AVRO) tag data and cache codecs; they are not security guarantees. JSON Schema $id is an identifier, not necessarily a network locator. OpenAPI Schema Object is a superset of JSON Schema 2020-12 with an OAS dialect. Alignments must record conflicts: JSON Schema is open-world constraints; Avro is a closed data definition with positional binary encoding.

1. What canonical form is used to decide whether two schemas are the same for reading, and which attributes are stripped? definition
2. What fingerprint algorithm and digest identify this canonical schema, and is it used as a cache key rather than as a security signature? identity
3. Which external standards is this contract aligned to, and which semantic conflicts are recorded instead of claimed conformance? interoperability
4. If string-encoded content declares contentEncoding or contentMediaType, is automatic decode disabled by default as JSON Schema requires for safety? security

Change proposal, review and approval

A change to a binding contract is a governed event, not a commit. It needs a proposal, an impact assessment against registered consumers, a compatibility check outcome, an approving authority appropriate to the severity, and a recorded decision. Semantic Versioning supplies the discipline that the released version itself is never edited: a correction is a new version. Where a jurisdictional profile applies, the change may also require re-checking conformance to that profile.

1. What events may trigger a contract change, and which of them are producer-initiated versus consumer-requested? event
2. Which registered consumers are affected by the proposed change, and how was that determined? relationship
3. Who approved the change, on what date and offset, and on what evidence? decision
4. What minimum notice period must elapse between approval and the change taking effect? temporal

Deprecation, sunset and retirement

Retiring a contract version is the step most often left undefined, and it is where consumers break silently. The contract needs a deprecation state distinct from retirement, a sunset instant after which the version is no longer served or supported, a migration path to the successor version, and a rule for what happens to data already produced under the retired version. DCAT-AP requires a controlled status vocabulary for distributions, and the Data Contract Specification's own deprecation in favour of ODCS with support stated through end of 2026 is a worked example of the pattern.

1. What distinguishes a deprecated version from a retired one in terms of what the producer still guarantees? state
2. At what instant does support for this version end, and is that instant published in advance? temporal
3. What migration path is offered to consumers, and who bears the migration cost? process
4. What happens to data already produced under a retired version - is it re-written, re-interpreted or left as-is? retention

Producer and consumer parties

A contract that does not know its consumers cannot assess the impact of a change. ODCS carries team and roles sections describing who is involved and what access each role holds. Registered consumers are the population that must be notified on deprecation and whose upgrade order matters under a compatibility mode. Unregistered consumption is a real and common state and must be modelled as such rather than assumed away.

1. Which party produces the data under this contract, and is there more than one producer? ownership
2. Which consumers are registered against this contract, and how is registration recorded? relationship
3. What role does each party hold, and what does that role entitle them to? access
4. How is unregistered or shadow consumption detected, and what is the policy toward it? exception

Obligations and acceptance

ODCS description.usage and description.limitations state what the data may and may not be used for; the Data Contract Specification's terms block covers usage policy, limitations, billing and notice period. Acceptance is the act that makes these binding on a specific consumer, and it has an effective period. An agent needs to distinguish an obligation the producer owes (structure, quality, freshness) from one the consumer owes (permitted use, attribution, no redistribution, notice on cessation).

1. What exactly does the producer commit to deliver, and in what form is that commitment testable? requirement
2. What restrictions bind the consumer's use, redistribution and derivation of the data? constraint
3. How does a consumer accept the contract, and over what period is that acceptance effective? process
4. What counts as breach by either party, and what remedy follows? exception

Consumer roles and access

ODCS roles list IAM role names, access mode (read or write), and first- and second-level approvers. JSON Schema readOnly and writeOnly describe instance properties managed exclusively by the owning authority or never returned on read. These are contract-level access terms, not a full IAM model.

1. Which roles may access data under this contract, with what access mode, and who approves grants? access
2. Which properties are readOnly or writeOnly, and what does the owning authority do if a consumer sends or requests them incorrectly? access
3. What is the default access rule when no role matches, and which break-glass exceptions exist? security
4. What decision record authorises a new consumer role binding to this contract? decision

Service level declarations and support

The Data Contract Specification defines servicelevels covering availability, latency, freshness and retention; ODCS carries a service-level agreement section plus support and communication channels. DCAT supplies dcterms:accrualPeriodicity for update frequency and dcat:temporalResolution for the finest temporal spacing. Each declared level needs a measurement definition, a measurement point and a named channel for raising a breach - a promise with no measurement definition cannot be breached or honoured.

1. Which service levels are promised - availability, freshness, latency, completeness of delivery, retention? measurement
2. How is each service level measured, at which point, and by whom? measurement
3. At what frequency is data updated, and what is the finest temporal resolution the data supports? temporal
4. Through which channel is an issue raised, and what response commitment attaches to it? process

Classification, personal data and usage restriction

ODCS carries a classification attribute on properties (for example confidential, restricted, public), a criticalDataElement flag and an encryptedName; the Data Contract Specification carries classification and an explicit pii boolean at field level. These declarations drive downstream access decisions but are not themselves access control: they are the input a policy engine consumes. Retention statements and jurisdictional restrictions belong here as declarations, with enforcement delegated to the access and records-management siblings.

1. What confidentiality classification applies to each property, and from which controlled vocabulary is it drawn? security
2. Which properties carry personal or specially protected data, and on what basis was that determination made? privacy
3. What protective measures does the contract require for classified properties - encryption, masking, tokenisation or pseudonym columns? security
4. What retention period and deletion obligation does the contract state for the described data? retention
5. Are there jurisdictional or residency restrictions on where the described data may be stored or processed? access

Server, format and serialization binding

ODCS carries an infrastructure and servers section and per-object physicalName and physicalType; DCAT expresses reachability through Distribution with dcat:mediaType and dcat:accessService, and DCAT-AP mandates IANA media types for dcat:mediaType and an EU vocabulary for dct:format. Iceberg treats partitioning and sort order as separately evolving physical concerns. The same logical contract may bind to several environments, and environment-specific differences must be visible rather than implied.

1. Which servers or environments does this contract bind to, and what distinguishes them? spatial
2. In what serialization format and media type is the data made available for each binding? interoperability
3. What is the physical name of each schema object in each target system? identity
4. How is the data partitioned or ordered physically, and is that part of the contract or an implementation detail? composition

Provenance and derivation of the contract artefact

The contract is itself an entity with provenance. PROV-O supplies the relations: wasGeneratedBy for the activity that produced it, wasAttributedTo for the responsible agent, used for the inputs consumed, wasDerivedFrom for derivation from another artefact and wasRevisionOf for a revision relationship. This matters practically because a schema inferred from sampled data carries a very different warrant than one authored from a governed specification, and consumers deserve to know which they are relying on.

1. Was this contract authored from a specification, inferred from data, or derived from another contract? provenance
2. Which agent generated this version, and was that agent human, automated or a combination? provenance
3. What inputs were used to produce this version, and are they retained for reproducibility? evidence
4. When was this version generated, and when was that generation recorded in the registry? temporal

Registry publication and discovery

A contract that cannot be found is not operative. A schema registry publishes under a subject determined by a subject name strategy and assigns a schema id and version. A catalogue publishes the governed dataset with a conformance link and, under DCAT-AP, a catalogue record whose application profile is declared. The two publication surfaces have different audiences - runtime serialisers versus human and agent discovery - and a contract usually needs both, with an explicit statement of which is authoritative.

1. In which registries or catalogues is this contract published, and which of them is the system of record? authority
2. Under what subject or record name is the contract published, and which naming strategy produced it? identity
3. How does a prospective consumer discover this contract and determine its applicability? access
4. How is divergence between the registry copy and the catalogue copy detected and resolved? quality

Schema language alignment and mapping fidelity

The same contract is routinely expressed in more than one language: JSON Schema for payload validation, Avro for wire format and evolution, SHACL for graph validation, ODCS for governance. These are alignments, not equivalences. Concrete losses are known: Avro records default-driven resolution that JSON Schema does not model; a JSON Schema format keyword under the format-annotation vocabulary is not an assertion at all; ODCS v3.1.0 added strict JSON Schema validation and relationships that older versions could not express. Claims of conformance to any of these require evidence, not assertion.

1. In which schema languages is this contract expressed, and which expression is normative? authority
2. Which contract facts are lost or weakened when projected into each target language? interoperability
3. What evidence supports any claim that this contract conforms to a named standard? evidence
4. Where do two aligned standards make contradictory requirements, and which one wins? exception

Profile and jurisdictional conformance

DCAT-AP 3.0.0 demonstrates how a jurisdiction narrows a base standard: it designates mandatory classes, marks properties as reused as-is, extended or profile-specific, mandates controlled vocabularies for named properties, and separates provider conformance from receiver conformance. A contract operating in a regulated or public-sector setting may have to satisfy such a profile in addition to its own rules, and profile obligations can differ for the party publishing and the party consuming.

1. Which jurisdictional or sectoral profiles apply to this contract, and on what legal or policy basis? authority
2. Which profile-mandated properties and controlled vocabularies must the contract populate? requirement
3. Do profile obligations differ between the publishing party and the receiving party, and how? classification
4. How is profile conformance demonstrated and re-verified after a contract change? validation