Debian Chooses A Reasonable, Common Sense Solution To Dealing With Non-Free Firmware

[Mattia_98]

IMO this is a bad thing to happen. They should have gone with the two images option. I myself use both images, depending on the situation. They already have two images one with and one without non-free firmware but they somehow decided to get rid of the free one.

[hwertz]

Back in the day when flash ROMs were common (I for instance had some older wifi cards with flash ROMs, SCSI cards flash ROMs were quite common, etc.), there were not significant problems with flash failures, these devices were not receiving 100s of updates, I doubt many were even flashed 10 times through the life of the device (some of the fancy SCSI/RAID cards with on board battery backed cache and etc. did receive quite a few updates both to fix bugs and to add features, but ordinarily one wouldn't flash every single update into it.) But, the fact of the matter is, the market has decided to prefer loading runtime firmware into RAM when possible.

I do find Debian's goal to have fully free software laudable, having the choice of having a system with open source BIOS, GPU firmware, wireless and bluetooth firmware, etc. would be lovely. But, given devices already running closed-source firmware, I don't see the distinction where having a driver load a blob into card RAM and then the driver interacting with that closed-source firmware is forbidden, while having a driver interact with potentially the exact same firmware but running out of a flash ROM is no problem. (Prism 2 wifi cards actually had this choice, you could flash updated firmware into the flash ROM, *or* have the driver load the same updated firmware into RAM on the card and it'd run the updated firmware from RAM rather than whatever firmware was in ROM.)

[oiaohm]

Flashable firmware has many more write cycles than you think.

No my numbers were not wrong. Flashable firmware is not created all the same. Its not uncommon for storage devices flashable firmware to only be able to be done 10 times include the 2 lost to factory testing firmware and the first written at factory firmware.

I stated the worse yes that is truly 10 cycle and the best is about 100,000 for flash storage in firmware. Remember your flash storage devices are using wear leveling and smart to be able to work around defective flash. Flash firmware without management software there risks here that flash firmware on device provide to customer is dud so will have very limited write cycles. Yes you can by a flash chip that say it has 100000 and it a dud and it turns out to have 10 write cycles. This is the problem for hardware vendors and flash firmware every time you tell customers to update firmware in flash you are rolling dice betting that the firmware chips they provided are to specification. One of the reasons vendors limit their updates is the fact flash is a dice roll.

Ram has lot lower defect rate and defects with ram are normally detectable straight away before device gets to customers. Vendors are way more willing to update firmware when it uploaded by drivers into ram on hardware because this is more reliable.

The reality here people get it backwards. Yes flashable firmware might have a lot of write cycles but when do you find out the real number of write cycles flash the answer is at the point it fails to write correctly. Flash device in specification saying it has 100000 write cycles is it most likely has that number.

Microsoft's policy is to have new "firmware" for Surface devices each and every single month.

Yes they did but about 3 percent of Surface devices failed on the 15 firmware updates no where near the 100000 write cycles expected. Yes Microsoft had mandated the 100000 write cycle class flash in the requirements. Flash like it or not is a dice roll Microsoft policy magically did not change this. All vendors using flash firmware without flash controller and do lots of firmware updates have end up on some products rolling basically snake eyes and having complete product lines fail after 10 to 20 firmware updates.

[oiaohm]

In MIDI, as in other hardware interfaces, if the firmware is abstracted entirely by way of the interface, there is no need for updates. Synthesizers don't need them. All of the samples in conventional synths are locked up in ROM as is the primary synth engine. The driver interface for GPU's should be standardized already: OpenGL, Vulkan, DirectX, etc. Move the hardware-specific driver code to the firmware and you solve the problem. There has never been a security issue with a synthesizer through MIDI commands.

MIDI does not provide direct direct memory access or with direct access to OS storage.

https://www.kaspersky.com.au/blog/equation-hdd-malware/7623/

Things get lot more complex from security side once you start having direct access to memory or storage.

Driver interface for GPUs are not standardized. Opengl. Vulkan and Direct X are all just abstraction layers. There is a driver layer under all them that converts the abstraction into what the GPU understand.

Please note you said conventional synths not all MIDI hardware synths are conventional. Yes you are right no MIDI synthesizer has had a security issue due to MIDI commands but some have totally bricked. Yes your non conventional reprogram-able MIDI hardware synths some of them have been lacking rom to be able to reset the device if something is wrong with the firmware and it happens to nuke it own eeprom. Yes it been some of these non conventional MIDI hardware synths that have end up bricked by using up all the eeprom write cycles as well.

Move hardware specific driver code into firmware mostly does not work with Vulkan and Direct X because these standards are still being changed. Opengl use to have this problem as well.

Basically there is a catch here items like Vulkan and direct X and networking yes lots of different areas the standards are not set in stone. So the physical hardware maybe able to accelerate X feature of Y standard but the board was released before X feature was added to Y standard so the firmware created when the board was created never exports the feature by the firmware API.

There are major reasons why you need to update firmware:
1)Security problems is you need to be able to update firmware.
2)Changing abstraction layers this is your Direct X and vulkan where the exposed API/ABI from the devices firmware end up not exporting everything that it need to now. With changing standards there is no way hardware vendor can have good enough view of future to get this stuff right at time of device being made.
3)Defects in firmware resulting in devices failing/miss behaving so requiring firmware replacement.

1 and 2 MIDI has mostly avoided. 3 MIDI does not. Items like GPU where the standards are changing things are absolutely not simple.

GPU area has all 3 problems
1)security faults turn up in firmware this is due to massive access with the DMA stuff even more with the direct to storage device stuff.
2) Yes the abstraction layer is always changing.
3) Defects with firmware happen this include power management mistakes that at times have been bad enough to completely kill GPU units.

Something like harddrive and storage devices have 1 and 3. So it depends on the type of device what reasons you will be needing to update firmware and how often. Devices that need firmware updated a lot the rom and ram combination is the safest. Yes brings the annoying problem that the OS has to have the firmware to send to device so device works.

[Waethorn]

No its not easy up-gradable on rewrite-able EPROM. All rewritable EPROM has limited write cycles some more than others the lowest write count for a re-writable EPROM is 10. Easy upgradeable is OS loaded firmware. This is small rom loader on card with ram to store the firmware. No write cycle limit here.

Do note lot of motherboards these days are including a rom loader to be able to rewrite the EPROM if a firmware update goes wrong.

The reality here is strict. There are really two valid options. ROM firmware loader/updater + ram or ROM firmware loader/update + EEPROM of some form.

The problem here you cannot have you cake and eat it. The ram option means the OS and it drivers have to provide the firmware so the device works but its the most durable option for items that do absolutely need security updates. EEPROM is the less durable option this end up discouraging sending out updates due to the write count limit. Socketing the EEPROM seams like a good idea until you look at socket failure rates vs soldered on.

Also that one of paying companies on going money to maintain the firmware only works if the company remains in business. There are many cases of people still using hardware well after the company that made the hardware is no longer in business. Subscription program could be part of the solution but I see the legally mandatory right to the source code of the firmware and keys to replace firmware if company either ceases to exist or decides not to provide updated firmware any more as the correct solution. Of course I would to say that the company has to give the updated firmware out for free if they are willing to maintain it so would allow them to run subscription model where those paying the subscription got the updated firmware and those not paying the subscription get informed they are using insecure. Yes signed firmware allows open source and subscription for firmware at the same time. Remember per device signing is possible.

Flashable firmware has many more write cycles than you think. Microsoft's policy is to have new "firmware" for Surface devices each and every single month. Mind you, not all of it is flashed firmware, but they can have UEFI capsule updates as often as they want. Other vendors limit their releases based on the severity of the bug. If the firmware interface restricts what the software can interact with, you don't have to worry as much about security or bug fix issues. The problem today is that concepts like UEFI make a whole C-programmed OS to exist in firmware, and for the software OS to interact in it in ungodly ways. It's layers upon layers, and a poor design, much like memory (CPU cache, with it's many levels) upon memory (RAM) upon memory (drive cache) upon memory (drive storage).

[oiaohm]

Resources and time are limited. Someone is going to find something a production team will have missed. This is why firmware is easily upgradable these days even if it's on a rewritable EPROM on the device. Planned obsolescence is an artificial problem separate from software maintenance though closely related in the form of withholding bug fixes for new purchases of hardware. It could be solved by a subscription program - pay us a living wage and we'll continue to offer the bug fixes. People have to eat, even FOSS developers. But I'm going to assume you want everything given to you as a freeloader from the tone of your rant.

No its not easy up-gradable on rewrite-able EPROM. All rewritable EPROM has limited write cycles some more than others the lowest write count for a re-writable EPROM is 10. Easy upgradeable is OS loaded firmware. This is small rom loader on card with ram to store the firmware. No write cycle limit here.

Do note lot of motherboards these days are including a rom loader to be able to rewrite the EPROM if a firmware update goes wrong.

The reality here is strict. There are really two valid options. ROM firmware loader/updater + ram or ROM firmware loader/update + EEPROM of some form.

The problem here you cannot have you cake and eat it. The ram option means the OS and it drivers have to provide the firmware so the device works but its the most durable option for items that do absolutely need security updates. EEPROM is the less durable option this end up discouraging sending out updates due to the write count limit. Socketing the EEPROM seams like a good idea until you look at socket failure rates vs soldered on.

Also that one of paying companies on going money to maintain the firmware only works if the company remains in business. There are many cases of people still using hardware well after the company that made the hardware is no longer in business. Subscription program could be part of the solution but I see the legally mandatory right to the source code of the firmware and keys to replace firmware if company either ceases to exist or decides not to provide updated firmware any more as the correct solution. Of course I would to say that the company has to give the updated firmware out for free if they are willing to maintain it so would allow them to run subscription model where those paying the subscription got the updated firmware and those not paying the subscription get informed they are using insecure. Yes signed firmware allows open source and subscription for firmware at the same time. Remember per device signing is possible.

[Waethorn]

This kind of states the core problem. Firmware for a lot of devices cannot remain in the chip. Issues of security issue with firmware do come up so it has to be replaced. Using flash on devices people put up as solution one problem flash does in fact have limited write cycles. Flashless devices have longer lifespan and being flashless is way less likely to be quirky.

Ram and ROM both can be made very reliability at low cost even after all these years we still cannot make flash at really low cost so a non updateable firmware solution on card is cheaper. ROM loader with ram based storage of firmware on card where the firmware has to be uploaded to make the device usable is cheaper.

Next if you have to replace the firmware in the field in a device a rom loader and ram base firmware storage card does has almost a zero risk of being bricked by a signed firmware upload. Yes vendors changing to signed firmware makes sense when you see return to manufactures.

Open source firmware does not straight up mean users can replace the firmware because the firmware binary still can be signed by vendor key so preventing user replacement.

The reality here is in a lot of cases it choice between OS loading firmware into device so you get firmware updates fixing hardware stability and security issues vs firmware written into rom on device and it never getting any updates ever and if the hardware has a defect having to replace the complete hardware. Replacing hardware due to minor issues is not good for waste.



This has a few mistakes. People have complained about MIDI music synthesizer this is why items like the preenfm that are fully open source and open hardware MIDI music synthesizers exist. The reason people don't complain as much about MIDI is that its a open standard and they have been able to make their own open source/open hardware alternatives.

Next big difference MIDI is a standard. Lets take Nvidia nightmare firmware for example. Nvidia firmware for there most recent cards turns out to expose interface yes but big problem its not standard so firmware is version locked OS driver version that understands the interface the firmware. Most recent is 20 series or newer. Nvidia is not the only one that does this Intel recently changed the interface firmware provided and attempted to change Linux kernel driver to only accepting the new one and failing with the old one and got told no way so now the Linux kernel driver will support both interfaces the intel hardware can provide.

Lack of standards to interface with devices is part of the problem here.

Yes problem here is making stable device interfaces also has problems with those that don't want to give away their intellectual property as well.

Next thing MIDI standard restricts the devices access. Firmware in a GPU, Network card, Harddrive.... all these directly connected devices can mess with your total OS security. Open source firmware does open up third party auditing of the firmware for security faults. Standard to interface with devices has advantages here particularly when it allows operating systems to limit the trouble devices can cause.

Not wanting to give away intellectual property turns out to being counter to having work audited in many cases.

This is not a simple problem as it first appears.

In MIDI, as in other hardware interfaces, if the firmware is abstracted entirely by way of the interface, there is no need for updates. Synthesizers don't need them. All of the samples in conventional synths are locked up in ROM as is the primary synth engine. The driver interface for GPU's should be standardized already: OpenGL, Vulkan, DirectX, etc. Move the hardware-specific driver code to the firmware and you solve the problem. There has never been a security issue with a synthesizer through MIDI commands.

[oiaohm]

Allowing closed-source firmware is a stopgap for hardware vendors that don't like the idea of giving away their intellectual property to write fully open-source hardware interfaces. IMO, firmware should remain where it belongs: on a chip, not in software..

This kind of states the core problem. Firmware for a lot of devices cannot remain in the chip. Issues of security issue with firmware do come up so it has to be replaced. Using flash on devices people put up as solution one problem flash does in fact have limited write cycles. Flashless devices have longer lifespan and being flashless is way less likely to be quirky.

Ram and ROM both can be made very reliability at low cost even after all these years we still cannot make flash at really low cost so a non updateable firmware solution on card is cheaper. ROM loader with ram based storage of firmware on card where the firmware has to be uploaded to make the device usable is cheaper.

Next if you have to replace the firmware in the field in a device a rom loader and ram base firmware storage card does has almost a zero risk of being bricked by a signed firmware upload. Yes vendors changing to signed firmware makes sense when you see return to manufactures.

Open source firmware does not straight up mean users can replace the firmware because the firmware binary still can be signed by vendor key so preventing user replacement.

The reality here is in a lot of cases it choice between OS loading firmware into device so you get firmware updates fixing hardware stability and security issues vs firmware written into rom on device and it never getting any updates ever and if the hardware has a defect having to replace the complete hardware. Replacing hardware due to minor issues is not good for waste.

Here's a question: why does nobody complain about MIDI in the same way? MIDI abstracts a music synthesizer's hardware through a set of common instructions. Only the control of hardware that is exposed via MIDI commands (notes, pitch bend, controller messages, etc.) and System Exclusive (SysEx) messages, as determined by the manufacturer are accessible. Everything else is hidden behind the shroud, hardcoded onto a few chips. You are bound to the limitations of the synthesizer by way of MIDI.

This has a few mistakes. People have complained about MIDI music synthesizer this is why items like the preenfm that are fully open source and open hardware MIDI music synthesizers exist. The reason people don't complain as much about MIDI is that its a open standard and they have been able to make their own open source/open hardware alternatives.

Next big difference MIDI is a standard. Lets take Nvidia nightmare firmware for example. Nvidia firmware for there most recent cards turns out to expose interface yes but big problem its not standard so firmware is version locked OS driver version that understands the interface the firmware. Most recent is 20 series or newer. Nvidia is not the only one that does this Intel recently changed the interface firmware provided and attempted to change Linux kernel driver to only accepting the new one and failing with the old one and got told no way so now the Linux kernel driver will support both interfaces the intel hardware can provide.

Lack of standards to interface with devices is part of the problem here.

Yes problem here is making stable device interfaces also has problems with those that don't want to give away their intellectual property as well.

Next thing MIDI standard restricts the devices access. Firmware in a GPU, Network card, Harddrive.... all these directly connected devices can mess with your total OS security. Open source firmware does open up third party auditing of the firmware for security faults. Standard to interface with devices has advantages here particularly when it allows operating systems to limit the trouble devices can cause.

Not wanting to give away intellectual property turns out to being counter to having work audited in many cases.

This is not a simple problem as it first appears.

[Waethorn]

i've had several Intel wifi chips where the final firmware was flakey, but that was it, the device was EOL so no new firmware.

Allowing closed-source firmware is a stopgap for hardware vendors that don't like the idea of giving away their intellectual property to write fully open-source hardware interfaces. IMO, firmware should remain where it belongs: on a chip, not in software.

Here's a question: why does nobody complain about MIDI in the same way? MIDI abstracts a music synthesizer's hardware through a set of common instructions. Only the control of hardware that is exposed via MIDI commands (notes, pitch bend, controller messages, etc.) and System Exclusive (SysEx) messages, as determined by the manufacturer are accessible. Everything else is hidden behind the shroud, hardcoded onto a few chips. You are bound to the limitations of the synthesizer by way of MIDI.

[hwertz]

I think I heard about these benefits, but is this such a common scenario when this stuff is urgently needed very often? I haven't heard about any specific case, like with specific old piece of hardware where this was beneficial. An example would be appreciated.

i've had several Intel wifi chips where the final firmware was flakey, but that was it, the device was EOL so no new firmware.