Nouveau: NVIDIA's New Hardware Is "VERY Open-Source Unfriendly"

[bridgman]

So once they were in that situation, how could they draw the line at "this firmware or this microcode has no source code and is too big", "this license is too restrictive", etc? There are no guidelines specifically for ethical firmware and microcode, so it ultimately was governed by existing free software guidelines by default.

Good summary. I think you hit the nail on the head with the last sentence. The purge started as a good idea (removing non-free *software* and microcode binaries with missing licenses or licenses which did not allow redistribution) and ended up going too far. Even that might be too strong a statement because it probably did result in a couple of network devices gaining open source firmware, but it's still hard to believe that the same rules should apply to "entire operating systems bigger than early versions of Linux" and, say, the microcode that allows our memory controller to train a GDDR5 link.

[Luke]

Microcode in flash or mask ROM no freer than on disk

It can be argued that the Intel approach of putting the microcode on a hard-to reflash flash chip or on a mask-programmed ROM is not only no freer than distributing it on disk, it is going backwards by removing the user's freedom to use or not use any alternative that might arise in the future-or the option to run the card with a particular function whose firmware becomes exploited disabled. For instance, suppose someone (NSA or spammer, it doesn't matter) develops an exploit against either AMD's video decode block or its firmware. If the firmware is on disk, it can be deleted/renamed and vdpau disabled until an update is released. If on the card this is harder to do and require a software layer to intercept and shut down calls to the offending module.

BTW, blacklisting all firmware and running VESA is NOT an open driver situation, as VESA is a wrapper around the (closed) video BIOS. Actually, being able to flash your driver (open or closed) into a bigger flash chip on the card along with any firmware used, then POST directly to it and use a "VESA 3D" wrapper to access it would be an interesting way to speed up boot processes.

Next up: open firmware on closed HARDWARE is still not "turtles all the way down" trust, if it was the three-letter agencies would not fear a foreign-made CPU in their own machines. OK, back to Intel. Traditionally they would publish a binary instruction set-but only in the earliest days if ever was this a bare hardware instruction sat.

I see three possible ways to make a CPU, GPU, etc-and as Bridgeman said this does NOT change how the design is written, only how it is manufactured after source code is written and compiled.

All instructions implemented in physical gates: cannot be updated, probably limited instruction set, potentially fastest and lowest power consumption. Instruction set to gates published, outputs knowable. Gates can be deduced if all possible inputs and outputs compared. Secret malicious gates possible, no firmware so no malicious firmware. Total trust possible only if entire HW design is known or determined by X-ray.

FGPA: all gates identical and known, all instructions implemented in firmware. Instruction set to firmware published. The firmware if closed should be possible to deduce as the gates are known. Umlimted instruction set possiblities, but largest die and slowest, highest power consumption. Malicious firmware w/ extra secret instructions possible, malicious gates less likely as instruction set cannot be predicted in advance by at the fab. Use with open firmware gives close to "turtles all the day down" trust. Possibility to create open HARDWARE, at price in power consumption on a warming planet.

Anything in between. Instruction set to firmware published. Common practice today, presumably cheaper and possibly smaller die than all instructions in HW. Updatable if a bug is found or a design improvement discovered. Comparing outputs and inputs can deduce equivalent FGGA or equivalent all hardware chip, but CANNOT deduce what is in firmware and what is in hardware. Both malicious firmware/hacked firmware via hex editor and malicous HW gates possible, open firmware plugs one risk but not the other.

In other words, open firmware would be useful on AMD cards (not being FPGA's with all known gates) mostly to allow a security audit, the driver devs presumably already have access to the firmware under NDA. This security audit would be incomplete without the hardware specs, reducing its value.

Thus, the open HW projects that so far have gone nowhere. Maybe FPGA chips could change this in the future, in the process replacing all that Boot Guard/locked devices crap with hobbyist boards that can become literally ANYTHING with the right firmware and open firmware existing to make things like Linux workstations out of them. Use another one for a gaming card or a supercomuting coprocessor if needed.

[chithanh]

We had this microcode discussion before in previous threads. It seems that some misconceptions have carried over here.

As I understand it, there are a few core arguments that come up:

These are all practical arguments. But the FSF has ethical reservations against proprietary software.

You may disagree with the FSF's point of view (I do), but it is valid and consistent point of view.

The situation with the free (in the FSF sense) distributions is that they will not be complicit in distributing proprietary software. And the Radeon microcode is undoubtedly proprietary software.

There are no guidelines specifically for ethical firmware and microcode, so it ultimately was governed by existing free software guidelines by default.

There are no guidelines because there are none needed. It is either distributed with source code and a free license, or it isn't. A few people from the Open Source camp (you know, the worst enemies of the Free Software movement) started arguing that stuff which does not run on the main CPU needs not be open source. But that argument is BS, it even has practical consequences. This was demonstrated by the 30C3 talk on the AMT keylogger and the following year's AGESA firmware hack. And the problems around the Atheros/Madwifi HAL exposed it as logically inconsistent and just used by proponents in an attempt to resolve their cognitive dissonance.

[boltronics]

It can be argued that the Intel approach of putting the microcode on a hard-to reflash flash chip or on a mask-programmed ROM is not only no freer than distributing it on disk

I responded to why this is not true earlier (and later gave practical examples of it already being a problem):

Actually, you're missing one of the core arguments; that being that the hardware vendor has more ability to control your hardware after the sale... if the microcode was permanently burned into ROM and could not be changed, this wouldn't be a problem - the manufacturer would be unable to issue a proprietary microcode update that rivals anything the user could create, because there is no facility to replace it.

If it's completely free, why would a vendor have more control over how the hardware can be used than you do? Who has the most control over your hardware is the key issue here, and if you read behind the lines in what RMS said to 4chan a couple of years back (where he specifically asks for microcode to be delivered via "ROM" - not "Flash"), I think you'll be able to pick up on that (although I wish he had explained this himself directly).

We had this microcode discussion before in previous threads. It seems that some misconceptions have carried over here.

Thanks for the links, as I hadn't noticed them previously. I'm glad I'm not the only one here asking about this. Again, the responses didn't seem to take into consider the above argument (and probably other issues not immediately obvious), which looks to largely be the cause of Bridgeman's apparent frustration.

These are all practical arguments. But the FSF has ethical reservations against proprietary software.

It almost sounds like you are suggesting there are no practical reservations, but that's ridiculous. There are always practical reservations driving the free software stance on ethics (going way back to the start with Stallman's issue with the printer driver), but they are not always obvious. In most cases, the FSF have had a great deal of foresight into potential issues and sidestepped them before they have become major issues. This has not always been the case (eg. I think they were somewhat slow to identify the SaaS threat, for reasons illustrated later in this post), but you can see from the Right to Read (which predicted Trusted Computing and the long-term significance of permitting DRM into society) that Stallman has done an amazing job of anticipating and responding to upcoming problems in general. However it's worth noting the trend that free software seems to solve all the problems, and so is probably the thing many automatically consider correct when in doubt.

And yet, as bridgeman and other people in those linked threads pointed out, the FSF seems to recognise devices with embedded firmware in ROM as ethical if they have free software drivers - presumably even if the firmware is many megabytes in size and is effectively running its own little operating system. That sounds to me they are *primarily* concerned with practical arguments! I agree - it's quite strange, but it's as if they are throwing their hands up in the air and saying "well it's stuck in ROM, so free software developers can't technically do anything about it, and the manufacturer can't change it either so nobody is oppressing the user by having more control, so there's nothing to be done here"!

But practical or no, I'm currently thinking that perhaps the core of this issue basically comes down to "when is microcode software?". Some would say it's anything that requires source code and a compiler, which perhaps excludes some microcode blobs due to the manufacturing process. Other people go to the extreme and say that basically anything stored in binary form, including documents, are software.

The Linux-libre project probably considers all microcode and firmware as "software". FSF generally probably think the same, but only if it's something the user manages (ie. installed with the distribution, the filesystem, or stored in any other storage capable of being written to by the user). I think the FSF would happily attack the manufacturers of proprietary firmware (when stored in writable flash) in HDD and SSD storage devices if there was something practical they could immediately do about it. Torvalds would probably technically have to disagree that microcode is software - it's embedded into his GPL'ed project (but perhaps not in files directly linked - would have to check how it's referenced). Bridgeman is somewhere in the middle, but thinks AMD's microcode is an example of something that shouldn't count as software, since it's created through basically the same kind of process as the actual hardware.

Does that view of the situation sound reasonable? If so, then it would seem to me that somebody in the free software community needs to come up with a formal definition (specifically to address this issue) as a first step to tackling this problem. I expect that would be formalised by somebody at the FSF, and whatever the FSF says would be treated as authoritative. As things stand today, I have never seen any documentation from the FSF strongly and clearly explaining their stance. Certainly RMS has explained what he believes the problem is, but not why he believes it's a problem, exactly where in the sand the line needs to be drawn, and the reasons why it should be that spot specifically. If they can clarify the position such that there no more confusion over the issue (we simply either agree or disagree), that would probably not only stop these kinds of discussions, but help motivate hardware manufacturers to meet specific requirements with a clear understanding of the reasons and advantages for doing so.

The situation with the free (in the FSF sense) distributions is that they will not be complicit in distributing proprietary software. And the Radeon microcode is undoubtedly proprietary software.

The Linux kernel contained binary microcode and firmware for many years. Perhaps this completely flew under the radar of the FSF (which is hard to imagine), or they decided they didn't want to deliberately publicise the problem too much at the time for some reason, or (most probably IMO) didn't immediately see it as a practical problem. What you describe may be their stance currently, but perhaps it's not (or it wasn't) as clear-cut as you make it sound.

When it comes to the FSF, all of their actions are basically predictable from looking at it like so: If you were RMS and you wanted to write the best possible printer driver for a new printer at MIT's computing lab today, what are all the problems that could stand in your way? Proprietary code, DRM, Tivoization, patents, licensing issues... everything they do comes down to that simple scenario. Now if the printer vendor said to RMS "here's the free software driver you can hack to your needs, to get it to work you just load this proprietary microcode into the printer and then you can create or modify the driver to perform this complete list of functions the printer is capable of", you might not immediately see a problem, which I think is why nothing was done about it for so long. But then imagine the printer manufacturer coming back the next day and saying "yeah about that microcode I gave you, I've got a new one here I'll sell to you for $<some large figure>, just load it through your driver and then you can do all these other new functions, oh and it fixes a bunch of problems you'll probably eventually run into as well", so although not immediately obvious, RMS later discovered it was not possible to write the best possible printer driver. He would also be pressured into relying on the manufacturer for non-free microcode. So once again, that situation with the printer, from which all FSF actions can be logically explained, indicates another problem the FSF must address.

A few people from the Open Source camp (you know, the worst enemies of the Free Software movement) started arguing that stuff which does not run on the main CPU needs not be open source. But that argument is BS, it even has practical consequences.

They say a lot of things. That's why I cringe whenever I hear the term "open-source". The vast majority of people using that term have no idea what the difference between it and "free software" even means. But even if there are practical consequences in an exploit of firmware on a ROM chip, I can't imagine it makes any different to the FSF (or anyone, really) if it can't be modified. It would mean the hardware must go back to the manufacturer, or the hardware would require replacing - but it would still have to be treated strictly as a hardware issue.

[chithanh]

It almost sounds like you are suggesting there are no practical reservations, but that's ridiculous.

Of course there is a practical side to this. But it is only of secondary importance. RMS said that using only free software is often the less convenient choice (ie. has practical disadvantages). Still one should do the ethical thing even if it causes an additional burden.

But practical or no, I'm currently thinking that perhaps the core of this issue basically comes down to "when is microcode software?".

That is easy to define. Computers are basically logic circuits, loading and executing instructions. When these instructions are reprogrammable, they are software. Otherwise, they are hardware.

[boltronics]

Of course there is a practical side to this. But it is only of secondary importance. RMS said that using only free software is often the less convenient choice (ie. has practical disadvantages). Still one should do the ethical thing even if it causes an additional burden.

That's a different story though. When they attack things (generally manufacturers or developers) - which is what we were discussing with regard to manufacturers and microcode, they always look at it from the angle of possible interference to practical application for the end user, since these are products designed for the end user.

However as for the recommendation that everyone should use free software (discussion which is generally focused on end users), that's completely different since you're likely not operating with the intent on pushing a solution onto other people. One could argue that the FSF pushes this recommendation (that you deprive yourself of a more convenient choice) so you won't be harmed - even in practical situations which will surely appear eventually - long term. However I think the main reason the FSF claims this is actually a matter of ethics is to help ensure you won't inadvertently harm other people attempting to use or switch to free software. eg. Say you use a proprietary chat program for features X, Y and Z since feature Z isn't supported by competing free software applications. Somebody else who wants to chat with you only cares about X and Y so would normally be able to use a free software chat program, but the two are not compatible so the free software user has to make the sacrifice for everyone to have the functionality they desire, etc. Same deal when collaborating with file formats, and many, many other very real situations. So in that space it actually *can* come down to a question of ethics. "Do I allow myself to use this proprietary program for feature X, even if it means it will encourage other people to adopt this program and thus allowing myself to harm other people?" (but in practice most people won't even notice or comprehend all the harm they are doing). So in short, these are two different things.

That is easy to define. Computers are basically logic circuits, loading and executing instructions. When these instructions are reprogrammable, they are software. Otherwise, they are hardware.

Your definition makes good sense to me too. But I feel we need to encourage someone at the FSF to make a clear statement around this.

This is interesting (taken from the front page of H-Node), the FSF's hardware database for free software compatibility:
"h-node lists only hardware that works with free drivers and without non-free firmware. Other GNU/Linux distributions (or Debian with contrib, non-free, or some 3rd-party archives enabled) include non-free firmware files, so they cannot be used for testing."

So they clearly only care about non-free firmware that is in the operating system, and not about non-free firmware on a hardware device in Flash. But this is inferred, not something they specifically point out. You would think this is exactly the sort of site the FSF would want to make the distinction on, although perhaps part of the reason for not doing so is to avoid making it too challenging for the end user to contribute - to discover if firmware is writable or not (which could be almost impossible to know for certain in some situations). Maybe that's why the FSF haven't cracked down on writable Flash-based firmware and microcode yet - it's currently too difficult for the end user to recognise?

I guess I'll just have to write them to see if I can get some answers.

[bridgman]

We had this microcode discussion before in previous threads. It seems that some misconceptions have carried over here.

I looked back and those threads before, but they all end up with something like "FSF says this because RMS says so, and I'm sure RMS has good reasons".

These are all practical arguments. But the FSF has ethical reservations against proprietary software. You may disagree with the FSF's point of view (I do), but it is valid and consistent point of view.

The situation with the free (in the FSF sense) distributions is that they will not be complicit in distributing proprietary software. And the Radeon microcode is undoubtedly proprietary software.

I have trouble believing that. Apart from it being a not-particularly-ethical position ("it's OK as long as we aren't associated with it") I have always heard more practical arguments from FSF, basically the ones I listed earlier.

If it's completely free, why would a vendor have more control over how the hardware can be used than you do? Who has the most control over your hardware is the key issue here, and if you read behind the lines in what RMS said to 4chan a couple of years back (where he specifically asks for microcode to be delivered via "ROM" - not "Flash"), I think you'll be able to pick up on that (although I wish he had explained this himself directly).

With respect, delivering via ROM (not PROM, EPROM or flash / EEPROM) is like delivering on papyrus scrolls these days. It's probably been 20 years since I last saw a masked ROM other than what gets built into chips directly.

You don't need to ban upgradeable microcode to prevent vendor control over your hardware, just put controls in the OS/driver and have enough spine to say "no" if you're offered cool new features that include something else you can't live with.

That is easy to define. Computers are basically logic circuits, loading and executing instructions. When these instructions are reprogrammable, they are software. Otherwise, they are hardware.

That's an interesting thought, although it doesn't jive with the FSF view that binary microcode in a flash is OK while the same microcode loaded by a driver is bad. I don't remember what their position was if the microcode was distributed in the flash but loaded by a driver included in the distro, but if you're talking to them maybe add that to the list.

If I understand your statement correctly, a microcoded device with a megabyte of code delivered via masked ROM on the chip would be "hardware", but the same megabyte of code delivered via flash or loaded into RAM would be "software" ?

How about the CPU case, where you have a large amount of microcode in masked ROM plus a small CAM that patches (over-rides specific words) the ROM'ed microcode ?

[bridgman]

Stupid edit limit.

[duby229]

Of course there is a practical side to this. But it is only of secondary importance. RMS said that using only free software is often the less convenient choice (ie. has practical disadvantages). Still one should do the ethical thing even if it causes an additional burden.

That is easy to define. Computers are basically logic circuits, loading and executing instructions. When these instructions are reprogrammable, they are software. Otherwise, they are hardware.

I don't know if I agree with your definition of what is software. The old versions of OpenGL for example was written to work on fixed function hardware. And yet OpenGL is definitely software. I don't think it would matter if it was flashed on die or not, or whether it was executed by fixed function hardware or programmable hardware, or whether or not that hardware is on die or off die.

[boltronics]

With respect, delivering via ROM (not PROM, EPROM or flash / EEPROM) is like delivering on papyrus scrolls these days. It's probably been 20 years since I last saw a masked ROM other than what gets built into chips directly.

Fair enough, but a PROM or EPROM (for example) would still technically suffice on consumer hardware, since it wouldn't reasonably be expected by a manufacturer that a user would ever be able to update the contents of those either. If the manufacturer can't get its code replaced by the end user, I expect the issue should still be sufficiently resolved.

You don't need to ban upgradeable microcode to prevent vendor control over your hardware, just put controls in the OS/driver and have enough spine to say "no" if you're offered cool new features that include something else you can't live with.

That would be a bit like the FSF saying "The FreeBSD license is good enough - if somebody forks a FreeBSD-licensed application and attempts to redistribute it as a proprietary application with new features, just don't use that particular fork"! They don't do things half-heartedly like that.

I don't remember what their position was if the microcode was distributed in the flash but loaded by a driver included in the distro, but if you're talking to them maybe add that to the list.

Good point.

If I understand your statement correctly, a microcoded device with a megabyte of code delivered via masked ROM on the chip would be "hardware", but the same megabyte of code delivered via flash or loaded into RAM would be "software" ?

That is my current thinking of how the FSF would interpret it, but I'll try to get clarification.

How about the CPU case, where you have a large amount of microcode in masked ROM plus a small CAM that patches (over-rides specific words) the ROM'ed microcode ?

Nooo!!!!! That opens up a whole new can of worms, doesn't it?

I would have to expect the patches (presumably stored on your HDD or other user-writable media - not sure what "CAM" is) would easily fall under non-free, unless the contents of the masked ROM was freed (so you could figure out what the bits in the patch were doing - or you'd otherwise effectively just have a binary diff). But in this way you would almost need to consider the masked ROM to be like Flash anyway (despite being extremely inflexible), so the masked ROM would need to be freed regardless of there being a patch or not.

I don't know much about masked ROMs (please point out if what I'm saying sounds stupid), but I imagine the ability to supply this patch is a function of the microcode or the chip itself? So maybe if the manufacturer does not want to release the microcode, it can just avoid adding that functionality?