FFmpeg Lands JPEG-XL Support

[coder]

In my head, I'm picturing rounding errors and different noise patterns at most.

That's why a good informal test is to subtract the images, bias & normalize the result, and then look at it to see if the noise has clear patterns to it or any resemblance to the original image.

If it were just rounding errors, then it should be pretty random. If there's real information loss, that's where you'd expect to see structure.

[brucethemoose]

I'd look at the data (i.e. PSNR or other measurements) and then decide whether the distinctions are significant. Also, I'm aware there are more sophisticated image quality metrics - I only use PSNR as an example.

Actually PSNR seems like a decent metric to start with... but maybe I am underestimating how significant those decoding changes would be. In my head, I'm picturing rounding errors and different noise patterns at most.

[coder]

Is PSNR going to have any meaning when the outputs are so close to each other? Even a max quality 4:4:4 jpeg will be massively lossy compared to those decoding differences.

I'd look at the data (i.e. PSNR or other measurements) and then decide whether the distinctions are significant. Also, I'm aware there are more sophisticated image quality metrics - I only use PSNR as an example.

[brucethemoose]

The second approach would involve starting with an original raw image. We could encode that to JPEG, then transcode the JPEG to JPEG-XL, and decode both. Now, compute the PSNR between the raw and the decoded JPEG. Do the same for the raw and the JPEG-XL transcoded version. Now, which has the lowest PSNR? It's conceivable that JPEG-XL could actually come out better, by virtue of a better chroma interpolation filter and/or more intermediate precision, assuming everything else (DCT blocks & colorspace) is the same.

Is PSNR going to have any meaning when the outputs are so close to each other? Even a max quality 4:4:4 jpeg will be massively lossy compared to those decoding differences.

I don't see a good way to test this with objective metrics... unless you simply average the diff, perhaps.

[quaz0r]

why argue over the use case of a 1Bx1B image? i think the only relevant point here is that no silly arbitrary limit was imposed, or if 1Bx1B is that limit, it was chosen to not be a limiting factor for anyone. how daft do you have to be to argue over the use case of the max possible theoretical thing? if i make a filesystem with no arbitrarily-imposed filesize limit, are you retards going to spend 8 pages of comments arguing over the use case and practicality of an infinitely large file? the autism struggle is real i guess

[coder]

Okay coder you win.

My goal is to learn and educate. Even if I turn out to be wrong, as long as it draws more quality information into the discussion, I figure we all win.

I don't pretend for a second that I'm without an ego, but if I remind myself that learning is what counts, then it's easier to accept being wrong about something.

Let's see if anyone actually do a more rigorous comparison of "do jxl converted from jpg output exactly the same image" to validate / invalidate the lossless transcoding claim by jpeg-xl.

I might dabble with it, myself. However, I first wanted to see if anyone with more JPEG-XL knowledge and experience would shed any light into these questions.

Which method shall we use? Convert both of them into bmp? Or do a printscreen for each of them?

I mentioned two approaches. First, one could decode both JPEG and transcoded JPEG-XL version to .BMP (or equivalent) and try subtracting them. Then, normalize the result and look for any color push in one direction or another. This is what I'd expect to see, if the transcoding involves any colorspace transform (or, more precisely, if JPEG-XL doesn't use the exact same YUV colorspace as JPEG). Perhaps it'd be best to try this on a file without EXIF or other metadata.

Next, I'd examine the diffs for quantized DCT artifacts around areas with large gradient magnitudes. If we see this in areas of constant chroma (most likely grayscale images), then that's pretty clear evidence they're indeed changing DCT sizes.

Finally, I'd look for chroma fringing around sharp color transitions, with images using 4:2:0 or 4:2:2 chroma sampling. If we see this but not luma atifacts, it would be clear evidence of the decoders using different chroma reconstruction filters.

I guess one more scenario is that differences are basically uncorrelated noise, which could be evidence the JPEG-XL decoder is carrying more intermediate precision between the stages.

The second approach would involve starting with an original raw image. We could encode that to JPEG, then transcode the JPEG to JPEG-XL, and decode both. Now, compute the PSNR between the raw and the decoded JPEG. Do the same for the raw and the JPEG-XL transcoded version. Now, which has the lowest PSNR? It's conceivable that JPEG-XL could actually come out better, by virtue of a better chroma interpolation filter and/or more intermediate precision, assuming everything else (DCT blocks & colorspace) is the same.

[billyswong]

Okay coder you win. Let's see if anyone actually do a more rigorous comparison of "do jxl converted from jpg output exactly the same image" to validate / invalidate the lossless transcoding claim by jpeg-xl. Which method shall we use? Convert both of them into bmp? Or do a printscreen for each of them?

[coder]

When a conversion is reversible, it is by definition lossless.

The existence of a path which can recover the original decoded output is effectively meaningless, if the path you actually use introduces differences. That's why I draw a distinction between reversibility and losslessness.

Furthermore, the state of reversibility seems affected by the tools. In time, bugs, fixes, or enhancements of other aspects could break the ability to exactly recover the original file. Therefore, I wouldn't depend on this path for lossless recovery of the original image.

Viewers or decoders are so inconsistent between each other.

Nearly all JPEG viewers use libjpeg or libjpeg-turbo. So, I wouldn't overstate the inconsistency issue, here.

I can still remember the same .mp4 looked different in brightness / gamma when in Quicktime in comparison to Media Player Classic within the same computer.

H.264 solved this problem by defining not only the compression syntax and semantics, but formally specifying decoder behavior. I think a lot of compression formats have followed, in this respect, as it's necessary not only to optimize image quality but also to combat error accumulation over long keyframe intervals.

Presumably, JPEG-XL has done the same. If anyone can confirm one way or another, it'd be good to know.

Factor in variation due to monitors, it is silly for one to expect even the same file can look the same to everyone in the world.

That's not the point of inquiry, here. It's merely to establish how "lossless" JPEG -> JPEG-XL transcoding is, since this seems to be a key feature of interest. How important losslessness is to someone is a subjective matter, and not something we can reasonably debate in the abstract.

[LinAGKar]

I use a normal release build and have that option.

I have that option too, but it doesn't seem like it does anything.

[billyswong]

Sure, there are differing DCT/IDCT implementations, as well as how chroma interpolation is performed, how much intermediate precision is carried forward to the colorspace transform, and how it rounds the results. Perhaps I was naive in assuming the reference JPEG-XL tools would treat each of these in a manner consistent with the original libjpeg.

One could control for some of that by using a monochrome or 4:4:4 JPEG file and potentially even skipping the colorspace transform - either by outputting native YUV or encoding in RGB (i.e. if the image isn't monochrome).

However, that would tell one less about what we actually want to know, which is the nature and significance of the differences between the original JPEG file and the JPEG-XL version. For that, it would be instructive to stick to more "typical" YUV 4:2:0 files and either analyze the differences between the two decoded outputs or ideally to compute PSNR vs. a raw original.

The potential benefit from this investigation could be informing users interested in maximizing the fidelity of their image collection that their best option would in fact be a full transcode.

I think you overcomplicated things. When a conversion is reversible, it is by definition lossless. When one (at least for myself) convert an archive of files from format x to format y, the major concern is are there artifact introduced by conversion. As long as no artifact are accumulated in the process, the conversion is safe.

Viewers or decoders are so inconsistent between each other. I can still remember the same .mp4 looked different in brightness / gamma when in Quicktime in comparison to Media Player Classic within the same computer. Factor in variation due to monitors, it is silly for one to expect even the same file can look the same to everyone in the world.