data processing
Let an agent explain data processing and its operations, relay methods, tools and standards from computing sources, describe the operations the registry aliases name, and distinguish data processing from data analysis, data science, information processing broadly and data protection in law.
Research draft, second pass
A second pass drafted this model: the structure a model of this thing needs, and what is known about it in the world. The line under this one says how the second half was obtained - researched against sources, or recalled without web access, in which case nothing here was read anywhere and every claim is a lead to verify. Unreviewed either way.
written by Claude from model knowledge without web access - no source was read, every claim is a lead to verify
Researched by: Claude
Purpose and description
Let an agent explain data processing and its operations, relay methods, tools and standards from computing sources, describe the operations the registry aliases name, and distinguish data processing from data analysis, data science, information processing broadly and data protection in law.
The collection, transformation and management of data by computers, from ingesting and validating input through operations such as serialization to storable formats, deduplication, normalisation and denormalisation of databases, access operations and round-tripping data between formats without loss, to output and reporting; data processing is both a set of processes and a field of work, historically associated with batch processing of business records.
What it is for: Turning raw data into usable information.
It can be explain the field; relay operations and tools; describe named operations; distinguish related fields.
Distinguishing features
Transformation of data
Pipeline operations
Field of work
Batch and streaming
What it looks like
Not a visible object; computational processes.
Physical character
historical form: batch processing on punched cards and mainframes note
modern pipelines: ETL and streaming list
legal sense: processing of personal data under GDPR note - distinct
How it is recognised
Collection and transformation of data by computers
Ingest, serialization, deduplication, denormalisation, access operation, data round-tripping
Analysis interprets data; data science models it; information processing is broader; data protection is legal
Related models
is a kind of - in registry terms
is a kind of - in registry terms
is contrasted with - which interprets data
is exemplified by - a common pipeline
In practice
Families and kinds
batch processing
real-time and stream processing
ETL and data integration
database operations such as normalisation and denormalisation
serialization and format conversion
deduplication and cleansing
Standards and regulation
Data format standards such as JSON, XML and Parquet
GDPR definition of processing of personal data
ISO 8000 data quality standards
Failure modes and hazards
Data loss in round-tripping
Confusing the computing and legal senses
Registry aliases mixing operations at different levels
Also called
+9
Where this came from
wikidata · CC0 1.0
Drafted structure
Bundle to layer to finding to question, as the second pass will find it: 4 bundles · 8 layers · 8 findings · 16 questions.
Understand What data processing is.
Definition.
Definition
Definition.
Definition
Definition.
- What is data processing, and how does it differ from data analysis, data science, information processing and data protection? definition
- Is the question about the field, an operation, tools or the legal sense? boundary
Operations
Operations.
Operations
Operations.
- What are ingest, serialization, deduplication, denormalisation, access operations and round-tripping? definition
- Which entry fits the specific operation? action
Methods Methods.
Science.
Pipelines
Pipelines.
Pipelines
Pipelines.
- How are batch, streaming and ETL pipelines designed? provenance
- Which references are standard? provenance
Quality
Data quality.
Quality
Quality.
- How are validation, cleansing and deduplication done? provenance
- Which sources are cited? provenance
Practice Practice.
Application.
Tools
Tools.
Tools
Tools.
- What tools and platforms are used for data processing? provenance
- Which entry fits data engineering? action
Law
Legal sense.
Law
Law.
- What does processing mean under data protection law? provenance
- Which entry fits data protection? action
Context History.
Context.
History
History.
History
History.
- How did data processing develop from tabulating machines to cloud pipelines? provenance
- Which entry fits the history of computing? action
Careers
Careers.
Careers
Careers.
- What roles work in data processing today? provenance
- Which entry fits data engineer? action
What the second pass must settle
- Should serialization and deduplication be separate primary entries?
- How should computing sources be linked?
- The registry entry has merged aliases naming specific operations; should they be split off?