Definition and the basic idea
Data compression is a set of techniques that reduce the number of bits needed to store or transmit data. The key principle is to replace the original representation with a more efficient one—usually by exploiting regularities such as repeated patterns, predictable sequences, or statistical imbalance (some symbols occur more often than others).
Simple model: encode the predictable, describe the rest
A helpful mental model is: (1) detect patterns, (2) create a smaller “description” for what you found, and (3) use that description to reconstruct the content at the other end.
Many compressors do this by transforming the data into a form that is easier to compress, then applying a coding step that assigns shorter bit patterns to more likely outcomes. A common structure is a combination of:
- A transform that reorganizes or predicts the data (so nearby values are related).
- A probability model that estimates what values are likely next.
- An entropy coder (a bit-level method) that turns those probabilities into compact bit strings.
If the reconstruction must be exact, the compressor can’t discard information; it must keep enough detail to reverse every transformation.
Two main types: lossless vs. lossy
Lossless compression reduces size while allowing perfect recovery of the original data. Typical targets are text files, program code, and many data formats where even a single wrong bit matters.
Lossy compression reduces size by removing information that is considered less important for the intended use. The original cannot be perfectly reconstructed. This is common for images, audio, and video, where small changes may be hard to notice depending on settings and content.
A crucial exception: even with lossy methods, the choice of how much to discard controls the trade-off between file size and quality. If the settings are too aggressive, artifacts become noticeable.
What compression is (and isn’t) good for
Compression is most effective when data contains redundancy (repetition or predictable structure) or skew (some outcomes are much more frequent). If the data looks essentially random, there may be little to compress—size may stay the same or even grow slightly because you still have to store compression metadata.
Also, compression isn’t the same as encryption. Compression changes representation to reduce size; encryption changes content to protect it. Sometimes systems apply both, but they serve different goals.
Practical checks you can do
To understand how a compressor behaves on your data, you can compare:
- Same input, different “quality” settings (for lossy formats) and observe the size/quality trade-off.
- Lossless vs. lossy on the same type of content to see whether exact reconstruction is required.
- Highly repetitive vs. random data to see how much structure your input has.
Uncertainty note: exact compression performance and best choices depend on the specific algorithm and your data characteristics, so results can vary even for the same file type across different sources.