IPU6 Firmware Binaries Upstreamed For Recent Intel Laptops

[phoronix]

Phoronix: IPU6 Firmware Binaries Upstreamed For Recent Intel Laptops

Webcameras on newer Intel laptops have been challenging for Linux use without resorting to an out-of-tree driver and proprietary user-space components, but that's been thankfully changing with progress being made on an open-source stack. There's still proprietary firmware necessary for enabling the IPU6 image processing unit, but at least that too is now in linux-firmware.git for easy distribution and packaging by Linux distributions...

https://www.phoronix.com/news/Intel-IPU6-Firmware-Git

[stormcrow]

FWIW, there's plenty of webcams out there that need a firmware blob that you may not be aware you even need because it's already installed in the linux-firmware archive. I'm not excusing Intel's actions here, merely pointing out they aren't alone on having loadable firmware blobs on advanced webcams. I'd prefer an open platform, if there were an open platform/specification for webcams with built in advanced image enhancement capabilities beyond exposure and focus (afaik there isn't), like most people concerned about FOSS & securable platforms. But it'd be demonstrably untrue they're the odd company out for webcams.

One of the things OpenBSD does when it installs a new system, if you have a webcam, it will automatically download the firmware blob file for the various webcams they know about and can redistribute the firmware for. Not all cameras need blobs, obviously, but it's a nice heads up and saves them from having to nitpick check every webcam against a hardware id list. Instead they just download one file regardless and let the drivers sort it out.

If you're interested what they have firmware for, go look here http://firmware.openbsd.org/firmware/ (latest version is 7.4, the file is uvideo-firmware). Download and extract it then look in the uvideo-info file for the specifics. Likewise you can do the same for the linux kernel at the link above.

[Adarion]

Yes, it is a bit puzzling that they do sound open firmare, but now people need FW for their webcams. I assume they just bought 3rd party equipment and now users have to fiddle around with the driver situation (remember Poulsbo and the likes, with the PowerVR-chips?).
FW is still a sore spot in the FOSS world.

[stormcrow]

This is true. But I think this is one of those cases where it's a big mistake to consider massive corporations like Intel an atomic whole. I think the pro-FOSS parts of the company are likely a small percentage of the whole, if important given the importance of high margin server parts to their bottom line. Those server parts are often, but not always, destined to run an open source OS whether it's Linux or BSD. Traditional desktop Linux is pretty much a blip in usage statistics. Polling commercial devs is only going to reinforce this since many commercial firms are Windows and if they have a server dev arm they're probably using either SSH to a headless dev machine, a VM Linux container, or WSL2. Android doesn't really count because firmware blobs and even driver blobs are more the norm than the exception, nor does Intel have a working phone hardware platform. That makes it less easy to make the pitch that FOSS benefits the balance sheet to overcome traditionalists that just want to hide everything on the mythical hypothetical that obfuscation and obscurity prevents IP theft (conveniently ignoring reverse engineering by disassembling the blobs).

[billyswong]

FWIW, there's plenty of webcams out there that need a firmware blob that you may not be aware you even need because it's already installed in the linux-firmware archive. I'm not excusing Intel's actions here, merely pointing out they aren't alone on having loadable firmware blobs on advanced webcams. I'd prefer an open platform, if there were an open platform/specification for webcams with built in advanced image enhancement capabilities beyond exposure and focus (afaik there isn't), like most people concerned about FOSS & securable platforms. But it'd be demonstrably untrue they're the odd company out for webcams.

The awkward situation of IPU6 webcams is they ask for more than binary firmware blob. They moved some software component from the embedded chip to the CPU, but didn't clear out the license/copyright issue of those code. Thus there comes closed-source unportable drivers again. Firmware blobs are OS-agnostic, therefore less concern if they are open source or not.

[samuelec]

I'm being naive but I don't get why Lenovo and thus Intel had to come out with those webcam in the first place instead of any other more supported.
It's a laptop webcam! when streaming a crap quality close to 480p will be the norm. Meanwhile 2 years are passed by and still can't use it, f* Lenovo

[jabl]

I'm being naive but I don't get why Lenovo and thus Intel had to come out with those webcam in the first place instead of any other more supported.

I guess it's a fraction a cent cheaper to have the "smarts" on a tiny corner of the chipset rather than a separate chip only for the webcam.

[stormcrow]

I'm being naive but I don't get why Lenovo and thus Intel had to come out with those webcam in the first place instead of any other more supported.
It's a laptop webcam! when streaming a crap quality close to 480p will be the norm. Meanwhile 2 years are passed by and still can't use it, f* Lenovo

Arguably if you're running Linux on a Lenovo laptop that originally came with Windows, you're not really Lenovo's target audience. They could just as well say eff you. Use it as we intended or don't come crying to anyone, but especially us, when things break.

That said, the reason why the IPU exists is Apple. Apple began making iPhones and iPads with cameras that could automatically process still and video streams in real time to significantly clean up pictures before they were saved to storage or sent over the Internet. It works pretty well. Then they brought it to their laptop line with the M series with a built in NPU. The current version of the IPU is Intel's answer to the Apple NPU based image processing capability. Like the Apple NPU, the current Intel IPU is on the CPU die. There are benefits to that arrangement in performance and power efficiency. It's not to save a few pennies on a separate dedicated processor. Assuming it works of course, I've heard it's problematic even on Windows, whereas the Apple image processing "just works".

480p hasn't been the webcam norm years now. Many people upgraded during the pandemic, almost all new webcams are 720p minimum or even 1080p. Or if one really wants, point your phone's main camera at yourself and use it with Camo (no idea if there's a FOSS equivalent) to get FHD+ depending on its resolution + its built in image processing capabilities. It's not just iPhones that have automatic inline image processing these days. Most newer Android phones have it too with varying degrees of usefulness. Newer smart phone cameras pretty much blow away any laptop webcam, including Intel's problematic IPU based laptops. Like many people, I never bothered with an Intel laptop for the past few generations. My desktop's webcam is a decent 1080p USB connected camera from Logitech I bought early on before the lock down rush. I forget the model. C9something, strangely it doesn't have a model stamp on the body. Works on fine in Linux. My Macbook Pro can either do the built in 720p with its image processing that runs on the M1's NPU, or I can just hang my iPhone on its back and use it through Camo which has a much better camera than all the above.

Camera fatigue is a thing. Poor quality images means people really don't want to pay attention to you, same with bad audio. Be excellent to each other. Don't be that person with the out of focus, too dark or too bright, washed out, low resolution camera from 2010 with a mic that makes you sound like you're in the bottom of a trashcan but still manages to pick up the least bit of sound for everything but your voice. This advice is from experience. If your ISP is utter crap, then that's generally beyond your control. Yet, it doesn't mean people can't do what's in their control, having a good video and audio setup is the bare minimum.

[samuelec]

Arguably if you're running Linux on a Lenovo laptop that originally came with Windows, you're not really Lenovo's target audience. They could just as well say eff you. Use it as we intended or don't come crying to anyone, but especially us, when things break.

We got those laptops for work and we use Linux unlike some other team, there's nothing I can do. Besides it's not the only laptops running this webcam neither we are talking about the crap realm of Apple products openness

That said, the reason why the IPU exists is Apple. Apple began making iPhones and iPads with cameras that could automatically process still and video streams in real time to significantly clean up pictures before they were saved to storage or sent over the Internet. It works pretty well.
...cut...
I've heard it's problematic even on Windows, whereas the Apple image processing "just works".

..and?

480p hasn't been the webcam norm years now.
.... cut ...
My Macbook Pro can either do the built in 720p with its image processing that runs on the M1's NPU, or I can just hang my iPhone on its back and use it through Camo which has a much better camera than all the above.

I'm using 4k webcam plugged into USB-c port and I wrote It's "like a 480p" when you are in video conference, although your picture looks amazing the other end gets much worse quality no matter what tech you got. Again, I don't care about your apple crap

Camera fatigue is a thing. Poor quality images means people really don't want to pay attention to you, same with bad audio. Be excellent to each other. Don't be that person with the out of focus, too dark or too bright, washed out, low resolution camera from 2010 with a mic that makes you sound like you're in the bottom of a trashcan but still manages to pick up the least bit of sound for everything but your voice. This advice is from experience. If your ISP is utter crap, then that's generally beyond your control. Yet, it doesn't mean people can't do what's in their control, having a good video and audio setup is the bare minimum.

Are you for real? Are you a person or a bot? Jeeez.. please, remember to take your medication