Pipeline, run and data provenance
Transformation definition, job, run, inputs and outputs, lineage at the dataset and field level. High-volume logs remain in external storage.
Research draft, second pass
A second pass drafted this model: the structure a model of this thing needs, and what is known about it in the world. The line under this one says how the second half was obtained - researched against sources, or recalled without web access, in which case nothing here was read anywhere and every claim is a lead to verify. Unreviewed either way.
Bundle → Layer → Finding → Questions Missing, in the backlog
Not described yet. This gap is in the card backlog.
Note: Research contour: bundles not designed yet; questions are listed as open questions.
Classifiers Filled
- Family
- Enterprise profiles
- Category
- Enterprise subject
- Entry kind
- subject
- Domain
- EnterpriseDAT
- Industry
- Cross-industry
- Tags
- EM-DAT-03W2subjectDataPipelinePipelineRunTransformationLineageAssertion
What it is Filled
Transformation definition, job, run, inputs and outputs, lineage at the dataset and field level. High-volume logs remain in external storage.
Why it exists Filled
Transformation definition, job, run, inputs and outputs, lineage at the dataset and field level. High-volume logs remain in external storage.
Distinguishing features Derived, awaiting review
- A run pins input versions
- Lineage has a source and a level of detail
- Absence of lineage does not mean absence of dependency
What robots and AI may and may not do Derived, awaiting review
Must not
- Negative case: Matching column names are treated as a proven transformation.
Note: Negative case of the research brief, not yet a rule for agents.
Moral aspects Missing, in the backlog
Not described yet. This gap is in the card backlog.
Owners Filled
Steward
Ответственный за данные / аналитическую функцию
Master systems
- Data catalog
- DWH
- BI
- оркестратор данных
Links to other meta-models Filled
neighbor
- EM-DAT-01
references
- WM-DAT-005 - conceptual-candidate
- WM-DAT-006 - conceptual-candidate
What else AI and robots need to interact with it Incomplete
Identity and identifiers required Derived, awaiting review
- DataPipeline
- PipelineRun
- Transformation
- LineageAssertion
Direct properties not applicable Not applicable
Not applicable
Enterprise record contour: physical properties belong to referenced world models.
Recognition required Missing, in the backlog
Not described yet. This gap is in the card backlog.
Capabilities and actions required Missing, in the backlog
Not described yet. This gap is in the card backlog.
Hazards and failure modes required Missing, in the backlog
Not described yet. This gap is in the card backlog.
Standards and interfaces required Missing, in the backlog
Not described yet. This gap is in the card backlog.
Context of use required Missing, in the backlog
Not described yet. This gap is in the card backlog.
Sources Filled
- W3C DCAT: dataset, distribution, catalog
- W3C PROV and Data Cube: provenance and observation
- SDMX and BI/data catalog practice: indicators, breakdowns, definitions and releases
Open questions
- How to link lineage without false causality?
- How to record a manual correction?
- How to reproduce a specific output?
- Установить границу и решение reuse/extend/new по действующим спецификациям.
- Подтвердить semantic crosswalk, права и source mastership.
- Выбрать immutable refs; провести проверки fixtures до заявления о публикационной готовности.
Machine files
Provenance
enterprise research programme · published-partial
Built from: enterprise/models/em-dat-03/brief.json
Still in Russian: suggested owner, blocking decisions, vercy candidates. Translate in research/enterprise/i18n/units.en.json.