JPEG
The usual account is that JPEG throws away pixels. It does not throw away a single one. It rewrites a block of 64 pixels as 64 numbers saying how much of each of 64 fixed patterns is present, rounds most of those numbers to zero, and rebuilds the block from what is left. Every pixel goes in and every pixel comes out; almost none of them comes out the same, and the picture still looks right. That substitution is the whole idea and it is visible in one block.
One block of eight by eight, and what survives being compressed
1 The image, in blocks of eight by eight
A block of eight by eight samples, which is what JPEG works on. Pick one with more or less fine detail in it.
The 64 original pixels, 0 to 255
2 The same block written as frequencies instead of pixels
The same 64 numbers, rewritten as how much of each of 64 fixed patterns is present. Top left is the average of the whole block; down and right is finer detail.
The 64 coefficients, before anything is thrown away
- the first coefficient
Nothing has been lost yet. This step is exactly reversible, and the page checks that when it loads.
3 Quantisation, which is where the throwing away happens
Each coefficient is divided by the number below it and rounded. That rounding is the only lossy step modelled on this page.
The quantiser, scaled to this quality
What is left after dividing and rounding
- rounded to nothing
- what is stored
4 Rebuild it, and look at what did not come back
The block rebuilt from what survived
- pixels that came back identical
- the worst any pixel is out by
- the average error
What this page checked when it loaded.
| claim | held | measured |
|---|---|---|
| the transform is reversible to within rounding | yes | worst difference 1.28e-13 |
| a flat block has exactly one non-zero coefficient | yes | DC 256.00, every other under 1e-9 |
| at quality 50 most coefficients are thrown away | yes | 59 of 64 became zero |
| and the rebuilt block is close but almost nowhere identical | yes | 24 of 64 pixels came back unchanged, worst error 4 |
| lower quality throws away more and costs more error | yes | quality 10: 60 zeros, rms 16.9; quality 90: 59 zeros, rms 0.4 |
What is real here, and what is not
One block, in grey, with no file around it
A real JPEG splits the image into colour and brightness, and most encoders then keep one colour sample for each two-by-two group of brightness samples before the transform starts, because an eye is worse at colour than at brightness. That is 4:2:0, and it is a common choice rather than part of the format: 4:4:4 keeps every colour sample and the standard’s own reference software supports it. Then this runs on every 8 by 8 block, and the result is Huffman-coded with the coefficients read in a zigzag so the zeros end up together. None of that is here. What is here is the one step where information is actually lost, on one block, and the compression that follows it is what turns those zeros into a smaller file.
The transform is computed from its definition, which no encoder does
Each of the 64 coefficients is worked out by summing over all 64 samples, which is 4,096 cosines per block and about as slow as it can be made. Real encoders use a fast algorithm that gets the same answer in a fraction of the work. Slow and exactly right was the correct trade here, because it means the sum written on this page is the sum in the code rather than an optimisation of it.
The quantisation table is copied, so it is checked against the standard
The table is the example luminance table from Annex K of ITU-T T.81. It is not normative — the standard offers it as one that has been found to work — and it is what almost everything uses. It is a memorised table, which is exactly the kind of thing that rots quietly, so the claims test compares all 64 entries against libjpeg-turbo's independent copy rather than trusting this file. The standard's own PDF is not machine-readable from the archive, and a second implementation is the better witness anyway: it would catch two rows exchanged, which a structural check would not.
Quantisation throws away small numbers, not fine detail
It is tempting to say JPEG discards high frequencies, and the fine stripes here show why that is wrong: they are the finest pattern the block can hold and they come back nearly intact, because their coefficient is enormous. What the quantiser removes is anything small relative to its divisor. A faint texture at the same frequency disappears completely; a bold one at that frequency survives.
Turning the quality up can make one block very slightly worse
The quality setting guarantees one thing and it is not the one people assume: no divisor in the table ever gets larger as quality rises, and that holds for all 64 entries at every step. What it does NOT guarantee is that any particular block comes out closer. A finer divisor can round a coefficient the other way, and on the gradient here the error rises slightly three times on the way from 5 to 100 before ending at zero. Over a whole photograph this averages out; on one block it does not, and the sweep on this page is one block.
Quantisation is the only lossy step HERE, not in every JPEG
The panel above used to say the rounding was the only lossy step in the whole format. On this page it is, because this page is one block of brightness with no colour in it. In a file encoded 4:2:0 the colour was thrown away before the transform ever ran, and that is lossy too. An outside reader caught the overclaim. The distinction worth keeping is between the mechanism this page demonstrates and the encoding choices it does not.
Sources
- ITU-T Recommendation T.81, Digital compression and coding of continuous-tone still images, 1992. The standard: the transform, and the Annex K example tables.
- libjpeg-turbo, jcparam.c. An independent copy of the Annex K luminance table, which is what the claims test compares this page's against entry by entry.
- N. Ahmed, T. Natarajan and K. R. Rao, Discrete Cosine Transform, IEEE Transactions on Computers C-23, 1974. Where the transform itself comes from, eighteen years earlier.
- Logical Art, the studio this belongs to.