Intel Skylake & Broxton To Require Graphics Firmware Blobs

[Kano]

You are really crazy! It is possible to update SSD firmware but it is not done every boot. A CPU usally gets microcode updates from the firmware but this could be done by the OS as well, especially some AMD systems really need those to fix some issues. But those are not critical to operate. You can not find any hardware without firmware - you just can select hardware that works with an OS that does not ship firmware without source code. If you use the CoreBoot modification LibreBoot you could get rid of binary blobs in the initial firmware but is is very limited to a few laptops (with replaced WLAN chip).

[Gusar]

Did you even care to check the log for these "magic" numbers? I did. All three show the same pattern. Initial release from Luis Rodriguez and subsequent fixes and clean-ups from openwrt developers (i. e., non-Qualcomm employees).

What exactly is your point?

Do you know what exactly these numbers do (beyond the generic "they set up the chip so it starts working")? And for example, if you had connection stability or slowness issues, would you be able to tweak these numbers to improve your wifi connection? Or would this require a dev to create new tables for you (I would have to check, I vaguely recall the application that creates at least some of these tables is open source, but still, would you know how to use it)?

Also, do you know what's in the firmware files for non-Atheros wifi chips? Here's a hint - lots of tables to set up the chip. The b43 firmware has files whose names contain "initvals" even! Just like those tables in the ath source are "initvals".
Kinda like the "golden registers" in the radeon source. Engineers at AMD told the open source team "set up the GPU with these registers and it'll be fast and stable". Why exactly those specific registers? Magic. Only known to those AMD engineers.

My point is this: The ath5k and ath9k drivers don't require files in /lib/firmware, but that doesn't mean the wifi chips are free of initialization routines that are magic to anyone but the wifi chip maker.

And all this does is further highlight the silliness of this outrage that Intel GPUs now require files in /lib/firmware. The lack of files in /lib/firmware for a specific device doesn't mean the device works without magic that is only known to the device vendor! So using the lack of files in /lib/firmware as a measure of "hardware free-ness" is, well, silly.

[PuckPoltergeist]

You are really crazy! It is possible to update SSD firmware but it is not done every boot.

So where do I get the source of this firmware?

A CPU usally gets microcode updates from the firmware but this could be done by the OS as well, especially some AMD systems really need those to fix some issues. But those are not critical to operate.

Often enough critical fixes are supplied with the new firmware. And again the same question: Where do I get the source?

You can not find any hardware without firmware - you just can select hardware that works with an OS that does not ship firmware without source code. If you use the CoreBoot modification LibreBoot you could get rid of binary blobs in the initial firmware but is is very limited to a few laptops (with replaced WLAN chip).

That's the point, you need the firmware for running modern computers. And most of it isn't free. So hardcore OSS purists don't have any option here.

[Del_]

Do you know what exactly these numbers do (beyond the generic "they set up the chip so it starts working")? And for example, if you had connection stability or slowness issues, would you be able to tweak these numbers to improve your wifi connection? Or would this require a dev to create new tables for you (I would have to check, I vaguely recall the application that creates at least some of these tables is open source, but still, would you know how to use it)?

Also, do you know what's in the firmware files for non-Atheros wifi chips? Here's a hint - lots of tables to set up the chip. The b43 firmware has files whose names contain "initvals" even! Just like those tables in the ath source are "initvals".
Kinda like the "golden registers" in the radeon source. Engineers at AMD told the open source team "set up the GPU with these registers and it'll be fast and stable". Why exactly those specific registers? Magic. Only known to those AMDgineers.

My point is this: The ath5k and ath9k drivers don't require files in /lib/firmware, but that doesn't mean the wifi chips are free of initialization routines that are magic to anyone but the wifi chip maker.

And all this does is further highlight the silliness of this outrage that Intel GPUs now require files in /lib/firmware. The lack of files in /lib/firmware for a specific device doesn't mean the device works without magic that is only known to the device vendor! So using the lack of files in /lib/firmware as a measure of "hardware free-ness" is, well, silly.

Those exact files you linked have gotten fixes and clean-ups from openwrt developers. I personally had to disable wireless-n on an Intel chip due to firmware issues. This information has already been given to you.

Moreover, if what you are saying was true, then opening the firmwares should be no problem for manufacturers.

Reality I am afraid is that firmware blobs vary from being huge blobs with large parts of the driver in them, to minor parts which could be considered parts of the hardware. In any case, as long as they are subject to bugs and bugfixing, they should be provided as open source. Lack of documentation or tools to generate them is of course unfortunate, but arguing that they'd just as well be blobs is ignorance in my book. It is also disrespectful to those we should thank for opening them up.

[Gusar]

Del_, you seem to think I'm arguing that vendors shouldn't bother working on opening stuff as much as possible, or that I don't value what Atheros in general, or Luis Rodriguez in particular, have provided. Don't know what exactly gave you that impression, or why you feel the need to be so overly defensive (it's kinda weird). One of my posts directly praises Atheros for the quality of their drivers!

I'm just saying, "magic" (stuff only the vendor understands) comes in various forms and is present also in seemingly fully open drivers. So yes, I'm still saying even ath9k contains magic, that a few Openwrt devs seem to know at least a bit about those tables doesn't change my view. But so what? The point is, it's silly to rage against a particular vendor regarding a particular form of delivering the magic. The focus should instead be on stuff like engagement with the community and working with the upstream kernel, not so much on ideological purity.

[schmidtbag]

re: the war metaphor
I prefer to have a map that anyone can see (also my enemies) than the one given to me by my enemies (or modified by some infiltrates of them) or the one that I can't even correct or write notes/shortcut on it!

Not sure you fully understood the metaphor. It takes the perspective of the developer, not the user. The map is the source code that you (the developer) created.

re: the car metaphor
I would buy a car with airbags developed by a good team of closed source developers only if also supported by the same company open driver or a community of open source developers.. it's all about choice, and closed source alone just give me obscurity (how the hell can I trust it?). But well first the most important things are the seat belts. And very horrible methaphor using fears (nobody loves car accidents) to manipulate the discussion.

I'm not saying you can't choose. I'm saying the obvious choice is the one that is most likely to succeed. If you want a safe reliable system, you go for the one with a paid devoted team. If that team happens to open source their code, great - even better. Usually, that's not the case.

The metaphor is actually very good because that extreme was (so far) the only thing that got you to not favor open source. The contrasts are much more apparent when its life or death. If I talked about something less interesting like ABS, I'm SURE you would pick open source. But the scale doesn't matter - ABS is still a risky feature to fail. Just because it's less intense it doesn't mean you should take it less seriously. Therefore, if you want to ENSURE safety and reliability for your computer, you go for the option that is the best supported. Again - if that is an open source option (which it can be) then great. But open source rarely has dedicated and obligatory developers, and doesn't always undergo thorough testing.

What about asking me know what a pedoterrorphile would prefer: an open source implementation of a browser+tor or IE? very low man...

The discussion is about whether closed source can be more reliable or trustworthy if put in the right hands. Web browsers are rarely critical applications and when they are, a firewall or antimalware is used as the primary defence. So the better question would be "what do you prefer, closed or open source antivirus". Having roughly 20 years experience repairing computers, I have yet to encounter a single free or open source antivirus tool that effectively protects a system. Also, IE is probably the absolute worst widely-used closed source application to have ever existed. It obviously didn't have a motivated dev team. I'm not vouching for just any closed software here, it needs to have a dedicated team.

re: calling me a religious
maybe we should had used the word fan boy.. that's a pretty offensive way to talk if you're out of names or silly metaphors. I just wrote "hundreds" of reasons without enlist them only because somebody already perfectly did.

I personally would say fanboys are a lower-blow. They're typically associated as being trolls and generally childish. You're serious and relatively mature. A fanboy would just simply say "closed source sucks" because of a biased opinion. You think closed source is actually detrimental. That's beyond fanboyism.

I found your way of defending closed source far more ideological than RMS battle for free software (who can afford saying something on the matter thanks to his previous large contributes to the community).

I'm not surprised you feel that way - I'm basically attacking your thought process. But the difference here is as much as it doesn't seem like it, I am a huge supporter of open source. I would love to see everything become open. HOWEVER, working in multi-billion dollar environments, I know why proprietary is necessary. For example, if something is free and open source and something goes wrong, who do you contact when literally every minute of downtime is costing your company thousands of dollars?

[horizonbrave]

@schmidtbag
Thanks for explanation, I understand much better your point of view now. I may not agree with some of it but I may acknowledge mine being lead by idealistic hope and not based from much experience. It's just sad to see how hard is for the FOSS model (and therefore Linux systems) to achieve progress in the desktop industry.. when I see so much efforts from "companies and agencies" for going in the opposite direction (it's normal I know) and I then naturally tend to disgust any closed source based software and hardly see as it could contribute to open systems growth

[Agross]

First, let's rebuke @schmidtbag propaganda.

As a software engineer with 15+ years of experience in C/C++ UNIX/Linux system/network programming I call bullshit : corporations hide behind 'closed source' to cover up abject untested and untestable shit written by 18 subcontractors in the 3rd world such that they maximize profit at YOUR expense.

GIven the ammount of breaches and failures of 'commercial' (closed soruce, for profit) software, the most glaring being the slowness, insecurity and brittlenes of the Windows family of operating systems, I seriously doubt how anyone can claim any notions of 'superiority'.

Most projects I've worked on, mind you, many were sophisticated distributed applications, had orders of magnitude less 'intellectual property' put into them by the software engineers than the open source compiler used to compile them and the open source operating system that, coincidentally, runs most of the 'multi-billion dollar environments' you are alluding to. From the critical network infrastructure

Most internal 'closed source' APIs I've seen ( if a company even goes that far to develop an API or framework ) are utter shit compared open source frameworks such as Boost, GTK and such. Most of them are untested, many are *untestable* (they are so poorly designed that they have to be refactored/rewritten in order to be be testable in an automated fashion )

How many bugs does M$ fix in a windows patch ? what did they cover ? nobody knows. At one project, once we open sourced a large framework for network control (used to manage 100k-s of machines ) and people started to use it, we had tonnes of bug reports, we had people commit hooks for other languages - scripting languages - that greatly improved testability ( software testers who do not know C/C++ or low level system/network programming could suddenly use Ruby to build complex test apps on top of it )

Also, the idea that open source has no support is false. There are companies that offer 24/7 support for many open source tools, frameworks, operating systems and such at a lower total cost than any 'commerical' software.

Wanna know why fglrx is so shitty ? Their 'secret sauce' is that they subcontracted parts of it to a well known ukrainian subcontractor know for its fast, cheap, shitjobs. I know because at one project we had to deal with low level shit written by a team of the said company who's members also worked on fglrx and the amout of crap I've seen in that code cemented the ideea that there is no way these people could write anything resembling a 'quality' code from any conciavable software engineering PoV.

With an open soruce tool, one can look at its code quality, bugs, bug fix rate and such. Given the Snowden revelations thant Microsoft was the first to jump in bed in 2007 with NSA and provides backdoors into its OS and Skype and the like, why would anyone use a closed source tool ?

For the mars lander to submarines, to power plants, linux - the free open source, community effort product - dominates. The ABS example is imbecile because 'hardware' production is not democratize as is software production and it comes from a different age and different mentality. It is not easy to mill your own transmission shaft. Maybe with 3D printing we will have open source hardware production along the lines of open source software production. If you look around the internet, there are open source beer and wine communities for with all the details of the production process are available and all the 'bug fixing' and 'features' are added in a transparent fashion. There are open source hardware companies that provide the entire blueprints and knowledge on how to build embedded platforms ( a-la Arduino ) and so on.

They don't open source ABS break design because they stand to loose in an open competition. Capitalists HATE competition, it eats away at their profits. The most profitable companies create (by chance or force) an artificial monopoly from which they derive 'monopoly profits'. If they loved competition so much, as they claim in their propaganda, all technologies would be open standards, where anybody could compete on an equal footing. Obviously they are not. And on most occasions, open source tools, APIs, protocols and such are far better than closed source / proprietary standards etc because they allow EVERYBODY to participate and add/use/tailor to their own problem, maximizing their usage, instead of enslaving them to one artificial 'standard' for the monopoly profit of one corporation.

The wireless spectrum is licensed because telecoms pour billions of dollars in the for of lobby ( which is a legalized form of bribery ) and because of the military connotations of the technology. We can't have open platforms telecommunication technologies because that would take away the monopoly profit of a handful of corporations that for a de-facto global oligopoly that dictates how/how much/where/whom can use a natural resource ( electromagnetic waves ). We affordable SDR (software defined radio) that could allow open source hardware telecommunication. It doesn't happen because your 'freely elected' representatives represent the interests of those who have the economic power in society. A man with 1.000.000 $ has more power than 1.000.000 people with 1$ each.


Now, for the uproar over Skylake : the first thing that comes to mind is the Orwellian silence of the 'technology enthusiast' community ( whatever the fuck that means ) about the nature of Skylake's most abject addition to x86 architecture : SGX ( https://software.intel.com/en-us/isa-extensions/intel-sgx ) the mother of all fucking DRMs in each CPU, that can execute arbitrary code and encrypt/hide arbitarary memory without the operating system ever knowing, all secured via a public-privatge key in the hands of Intel (each of your marvelous Skylake CPU will have a set of private keys ). One will need a multi-million dollar installation of considerable technological sophisitication to get rid of this by physically modifying the CPU die. They need closed source blobs on linux because the path from the cpu/memory/dma to the gpu is encrypted, because the path from inpud/disk/dma is encrypted ( both outside OS control ) and there is no way in hell to open source them.

If you are just a little bit technically inclinded, read throughout the programmer's guide on Intel's site and see what an abject shit SGX is. Then we can debate how come every 'technologist' with a fancy website that vomits benchmarks, menchmarketing and straight-out marketing propaganda from various corporations (from 'redutable' Ars Technica to some 'cartoony' Fudzilla - both of which regularly talk about Snowden, CIA, 'security' and such ) managed to miss the implications of SGX ...

Welcome to the brave new world where Intel, CIA, MPAA and the like run arbitrary encrypted code on your machine and there is nothing what so ever you can do about it. WHy do you think the russians and the chinese are building their own CPUs ? This is not for *your* protection, their biggest customers are Microsoft, Apple, Google, banks and so on.

[Del_]

HOWEVER, working in multi-billion dollar environments, I know why proprietary is necessary. For example, if something is free and open source and something goes wrong, who do you contact when literally every minute of downtime is costing your company thousands of dollars?

Are you serious? You are aware that business critical solutions in multi-billion dollar environments have embraced open source more than just about anybody else? Let me answer your question simply. If something goes wrong, you are free to choose between multiple providers (e.g., Red Hat *and* Suse) when it is open source. You can even have your own people fix it. With closed source you are stuck with the people responsible for the problem, and are completely at their grace. Open source removes vendor monopoly.

[Kano]

I don't think that there are so many ppl interested in fixing drivers on their own. Reporting bugs yes, but look at yourself bis offen you fixed something. I did that only for very small issues, one little hardware preset in mesa (for a radeon OEM card), a few missing PCI IDS or simple alsa hacks which just store some specific model overrides. For many other things you have to dig much deeper. If you can directly contact devs fixes could be done of course much faster - i tried about 3 or 4 ways to report fglrx bugs, registered at least 3 accounts just for that and the result... Better forget it.