Rav1e Sees New Pre-Release With More Speed-Ups, Monochrome Support

[phoronix]

Phoronix: Rav1e Sees New Pre-Release With More Speed-Ups, Monochrome Support

Since the release of Rav1e v0.3.1 back in February the "weekly pre-releases" dropped off until this week with there finally being a new tagged milestone...

http://www.phoronix.com/scan.php?page=news_item&px=Rav1e-May-Speed-Ups

[microcode]

Sweet, I was hoping for monochrome to come.

[milkylainen]

A side note, a highly subjective one and totally beside the topic.
But there is not a single encoder code-base that I have looked at that does not look like horrid shit, rav1e included.
Building it in Rust does not make this aspect better apparently, in any way or form.

Or maybe it is just me?

[cynic]

Sweet, I was hoping for monochrome to come.

can't tell if you're being sarcastic or not

anyway, I was wondering what are the usecases for monochrome.

[starshipeleven]

A side note, a highly subjective one and totally beside the topic.
But there is not a single encoder code-base that I have looked at that does not look like horrid shit, rav1e included.

What do you mean exactly with "horrid shit"?

[bug77]

What do you mean exactly with "horrid shit"?

Doesn't read like (pardon the expression) JS

[microcode]

can't tell if you're being sarcastic or not

anyway, I was wondering what are the usecases for monochrome.

One common use case is near-infrared surveillance footage. I also share monochrome videos regularly, for the art of it.

[schmidtbag]

Slightly off-topic, but I think it'd be good for OpenCV to support true monochrome. As far as I'm aware, single-channel monochrome must be converted to BGR. I understand OpenCV probably wasn't built for single-channel monochrome, but sometimes all you need is to detect shapes or shades, or perhaps you're using an IR camera. Having 1/3 of the data would dramatically decrease processing.

[polarathene]

I was wondering what are the usecases for monochrome.

Reduced data or higher quality data iirc. If you want to send a stream of data values such as a depth/height map for use as a texture on a 3D mesh, you only need the single channel, and chances are if you try to pack any other data, eg other single channel data streams, then you'll probably find the compression algorithms assuming it's RGB data and ruining it for processing due to artifacts.

Similar can be found with texture compression codecs with 4-channel HDR images where the alpha channel is providing a multiplier or exponent for RGB values to go from 8-bit to 10-bit or higher. The texture compression algorithms look for surrounding blocks of RGB colour to compress, and don't recognize that alpha is not correlated colour/alpha wise and should be encoded as a separate plane. The solution there at least for ASTC was to pre-process the alpha channel to be roughly the same value of neighbouring pixels in a block and adjusting their RGB pixels to roughly equate to the desired output value when paired with the new multiplier/exponent. Normal Maps is another one where the data is non-colour, you only need two channels for it, but texture codecs may not handle it well, so general approach is to encode as two separate textures/planes.

One case with video was 8-bit wasn't really enough of a range 0-255 for a height map. We had a flat 3D plane(as in geometry), which in real-time is projecting triangles forward into the 3rd axis to represent frames of the actual 3D shape. That was streamed as a video as a cheap way to produce "3D" video (from a limited field of view) via depth capture devices like Microsoft Kinect or Intel Realsense. So you'd be wasting 2 extra channels duplicating the same data. If you try to use the other channels for increasing the bit-depth it munges the data iirc (because as said, it assumes colour data that correlates between channels, not high/low 8-bits intended for 16-bit image).

There's other workarounds like doubling the frame size, which either reduces your resolution or increases the bandwidth and processing, which isn't always viable as some hardware or codecs can have limits. Eg h.264 can only manage up to 4K iirc, and some mobile devices can only handle 4K or 8K textures dependent on the GPU. This was years ago when I did it, I think h.265 wasn't an option as support for it wasn't as widely available, that'd cut bandwidth in half, among other benefits, and h.264 10-bit support(1024 values vs 8-bit 256) wasn't available in the video decoder(UnrealEngine 4) at the time at least.

One issue I guess you'd still run into with monochrome streams is if you need to synchronize the frames if you cannot get enough bit-depth and need to composite multiple streams, or for a similar reason multiple streams are related(streaming data textures for same model). If there's resolution budget to spare in the video texture, then you could as mentioned earlier, combine two streams into one, with their frames playing side by side in sync.

Tried to look up UE4 docs on video codec support, but they seem a bit vague, afaik h265 still isn't supported, possibly because of licensing fees, that supports up to 16-bit monochrome, but if AV1 gets picked up 12-bit is still good, maybe for UE5 by 2022

[milkylainen]

What do you mean exactly with "horrid shit"?

I suspected this. I don't want to get into a flame war over subjective things.
But like .. the usual suspects?
Code structure, indentation, non-obvious/hiding control flow, naming of variables and symbols.
Just plain fugly in my eyes. Not limited to rav1e of course.