data compression
Let an agent explain data compression and its methods, relay theory, algorithms and applications from computer science sources, describe the techniques the registry aliases name, and distinguish compression from encryption, deduplication, archiving and probabilistic data structures.
Research draft, second pass
A second pass drafted this model: the structure a model of this thing needs, and what is known about it in the world. The line under this one says how the second half was obtained - researched against sources, or recalled without web access, in which case nothing here was read anywhere and every claim is a lead to verify. Unreviewed either way.
written by Claude from model knowledge without web access - no source was read, every claim is a lead to verify
Researched by: Claude
Purpose and description
Let an agent explain data compression and its methods, relay theory, algorithms and applications from computer science sources, describe the techniques the registry aliases name, and distinguish compression from encryption, deduplication, archiving and probabilistic data structures.
The process of encoding information using fewer bits than the original representation, either lossless, allowing exact reconstruction through entropy coding and dictionary methods, or lossy, discarding less important information as in JPEG and MP3, including executable compression, solid compression of many files as one stream in archivers, dataset compression and model compression of neural networks by pruning and quantisation; approximate membership query filters such as Bloom filters are related space-efficient data structures rather than compression per se.
What it is for: Reducing data size for storage and transmission.
It can be explain lossless and lossy compression; relay algorithms and theory; describe named techniques; distinguish related concepts.
Distinguishing features
Fewer bits
Lossless or lossy
Entropy bounds
Ubiquitous
What it looks like
Not a visible object; smaller encoded files.
Physical character
Shannon source coding theorem: 1948 year
Huffman coding: 1952 year
LZ77: 1977 year - Lempel and Ziv
JPEG standard: 1992 year
How it is recognised
Encoding data in fewer bits
Entropy coding, executable compression, solid compression, dataset and model compression; AMQ filters as related
Encryption hides content; deduplication removes copies; archiving bundles files; Bloom filters answer membership queries
Related models
is a kind of - in registry terms
is bounded by -
includes -
is contrasted with -
In practice
Families and kinds
entropy coding such as Huffman and arithmetic coding
dictionary methods such as LZ77 and LZW
lossy media codecs
executable compression
solid archive compression
model compression for neural networks
Standards and regulation
Codec standards such as JPEG, MPEG and zlib RFC 1950
Patent and licensing regimes for codecs
Failure modes and hazards
Quality loss in lossy compression
Compression bombs
Registry aliases mixing compression and filters
Also called
Where this came from
wikidata · CC0 1.0
Drafted structure
Bundle to layer to finding to question, as the second pass will find it: 4 bundles · 8 layers · 8 findings · 16 questions.
Understand What data compression is.
Science.
Definition
Definition.
Definition
Definition.
- What is data compression, and how does it differ from encryption, deduplication, archiving and probabilistic filters? definition
- Is the question about theory, an algorithm, a format or a specific application? boundary
Techniques
Named techniques.
Techniques
Techniques.
- What are entropy coding, executable, solid, dataset and model compression, and AMQ filters? definition
- Which entry fits the specific technique? action
Theory Theory.
Science.
Entropy
Information theory.
Entropy
Entropy.
- How does Shannon entropy bound lossless compression? provenance
- Which references are standard? provenance
Algorithms
Algorithms.
Algorithms
Algorithms.
- How do Huffman, arithmetic, LZ and transform coding work? provenance
- Which sources are cited? provenance
Applications Applications.
Application.
Media
Media codecs.
Media
Media.
- How do lossy codecs for images, audio and video work? provenance
- Which entry fits JPEG? action
ML
Model compression.
ML
ML.
- How are neural networks compressed by pruning, quantisation and distillation? provenance
- Which entry fits quantization (machine learning)? action
Context History and security.
Context.
History
History.
History
History.
- How did compression develop from Morse code to modern codecs? provenance
- Which entry fits the history of data compression? action
Security
Security issues.
Security
Security.
- What risks such as compression bombs and side channels exist, at a general level? provenance
- Which entry fits zip bomb? action
What the second pass must settle
- Should model compression and solid compression be separate primary entries?
- How should computer science sources be linked?
- The registry alias approximate membership query filter is a data structure rather than compression; should it be split off?