Khronos Establishing A Camera API Working Group

[coder]

I was recently working with video4linux and opengl on nvidia and there wasn't any efficient way to display the camera with opengl, and the next best alternative was to write a lot of shader code to convert image formats at real time.

Try using a media-processing framework, like GStreamer or ffmpeg. They have color-conversion filters. You can even find GPU-accelerated ones,

That being said, doesn't NVidia have some extension for YUV 4:2:0 textures?

Hopefully this new api works nicely with opengl (which I assume it should because its khronos we are talking about).

Not necessarily. It seems to be pretty low-level. Maybe there'll be some interop extensions showing up in OpenGL, if only vendor-specific ones.

you still need to convert image formats at real time and write that code yourself in a shader.

You don't have to do it in a shader. In a lot of cases, using the CPU is still plenty fast.

[coder]

this will be nice when it materializes, khornos has a strong track record if memory serves correct.

Eh, it's hit-and-miss. There's a list of "inactive standards" at the bottom of this page (with probably a few more headed in that direction):

https://www.khronos.org/developers

Interestingly, one that I'm seeing is OpenKCam, "an open, royalty-free standard for advanced, low-level control of mobile and embedded cameras and sensors". However, it seems they never managed to finalize it. I'd imagine it's mostly a subset of what the new standard is aiming to address. And dec05eba, that one indeed states that one of its goals was:

Efficient interoperation with, and fulfillment of key use cases for, OpenVX, OpenGL, OpenGL ES and OpenCL when used for image and vision processing.

Anyway, Khronos is just an industry consortium. It and its working groups are run by member companies, so inactive standards should probably be seen as more a statement about insufficient industry demand, in most cases. In others, they might've simply gotten beaten out by alternative standards or solutions.

[dec05eba]

That being said, doesn't NVidia have some extension for YUV 4:2:0 textures?

I tried using that and if i recall correctly, it didn't work on nvidia desktop gpus and only works with tegra? tegra also supports dma so tegra has all the nice things that desktop gpus dont have. I dont think that extension is available for intel/amd either so it wouldn't be cross platform.

You don't have to do it in a shader. In a lot of cases, using the CPU is still plenty fast.

I have done it on the cpu as well, and its not fast enough. It takes several milliseconds but it cant take more than 1-2 millisecond. Of course there is a solution (mpv does it), its just that it requires much more work than it should when you just want to have a camera feed with almost no cpu usage.

[Quackdoc]

Eh, it's hit-and-miss. There's a list of "inactive standards" at the bottom of this page (with probably a few more headed in that direction):

https://www.khronos.org/developers

Interestingly, one that I'm seeing is OpenKCam, "an open, royalty-free standard for advanced, low-level control of mobile and embedded cameras and sensors". However, it seems they never managed to finalize it. I'd imagine it's mostly a subset of what the new standard is aiming to address. And dec05eba, that one indeed states that one of its goals was:


Anyway, Khronos is just an industry consortium. It and its working groups are run by member companies, so inactive standards should probably be seen as more a statement about insufficient industry demand, in most cases. In others, they might've simply gotten beaten out by alternative standards or solutions.

I see regardless I hope this works out.

[Zoll]

As someone who writes camera drivers for a living, I welcome this change IF it gets major industry support across all platforms and architectures. It might literally take decades for the industry to converge to a common standard, but someone must start somewhere.