DragonFlyBSD's Meltdown Fix Causing More Slowdowns Than Linux

[dillon]

Thanks for running those benchmarks testing our Meltdown mitigation, Michael! The results are about what we expected. The more system-call-heavy an application is, the more it suffers from the ~150 to ~250nS of additional system call / interrupt / exception overhead. The less system-call-heavy, the less it suffers.

Someone asked about PCID. DragonFlyBSD does not currently implement PCID so the results you are seeing are from two %cr3 loads without PCID in the system call path. I don't know what the difference would be... PCID didn't actually do a whole lot when we tested it a year or two ago, because it's only applicable for major process context switches and those are already well-managed by the scheduler. (Well, and virtualization too, but virtualization is not DragonFly's strong point). However, I will be looking at using PCID again to potentially reduce the impact of this mitigation, since Meltdown on Intel requires the MMU to be reloaded twice in these heavily used code paths.

The linux guys are saying that Meltdown losses are roughly cut in half with PCID in these code paths. That makes sense. But its still bad.

IBRS (one of the Intel Microcode mitigations) is even worse. If you enable IBRS at all times (that would be linux ibrs mode 2), performance is completely destroyed for just about everything. We're talking 30% or worse. For everything. IBRS has to be left on all the time to protect against two types of Spectre attacks --- same process (e.g. javascript -> client browser) attacks, and sibling hyper-thread attacks. I consider Intel's microcode fixes to be a non-starter, honestly. They are unusable. I'll put support in DragonFly anyway, soon, but I consider it worthless. Spectre will not be truly fixed without new silicon... probably 6 months to a 12 months away, or worse.

With IBRS, system call overhead increases from 380ns to 780ns on the one machine I have been able to test it on so far (an older Haswell), and performance is lost across far more than just the system call boundary.

The partial IBRS mitigations across the user->kernel boundary provided by Intel are... I'm still vomiting.

-Matt

[dillon]

One other note, for DragonFly users, we have a Wiki page with instructions on how to run the Chrome browser securely (or as securely as its possible to run it) that we recommend users use. A lot of this is applicable to Linux and the other BSD's, too. Basically, two levels of segregation. The first is you segregate the browser from your main account by creating secondary user ids and home dirs that are ONLY used to run the browser (and use ssh to script the startup). The browser is still fast as one can still make a direct X11 connection (but if you want to forward the X11 over ssh, or disable /dev/dri/card* acceleration, that works too and is somewhat more secure). The second is that chrome itself has an experimental option called --site-per-process which honestly everyone needs to be using.

https://www.dragonflybsd.org/docs/docs/handbook/RunSecureBrowser/

-Matt

[F.Ultra]

Seriously? You didn't hear about Snowden? All the people at NSA save one knew what they were doing and kept their mouth shut. How is this even comparable as a moral dilemma? A "sanctioned/required" backdoor in CPUs is magnitudes lower on the moral radar.

Well Snowden didn't keep silent now did he? And he is far from the single NSA whistleblower. Still keeping secrets within an organisation built for such (as the NSA) are orders of magnitude easier than keeping a lid on an entire chip industry (Intel, ARM, nVidia and so on) where even some like ARM are outside the US jurisdiction. Furher there where no leaks of tools or even hints at such tools existed to utilize these backdoors in processors in any of the documents leaked by Snowden and others.

And that we have people in this very thread that apparently still believes that Kennedy was a conspiracy just shows that some people have a need to "believe".

[Almindor]

Well Snowden didn't keep silent now did he? And he is far from the single NSA whistleblower. Still keeping secrets within an organisation built for such (as the NSA) are orders of magnitude easier than keeping a lid on an entire chip industry (Intel, ARM, nVidia and so on) where even some like ARM are outside the US jurisdiction. Furher there where no leaks of tools or even hints at such tools existed to utilize these backdoors in processors in any of the documents leaked by Snowden and others.

And that we have people in this very thread that apparently still believes that Kennedy was a conspiracy just shows that some people have a need to "believe".

I wonder if you'd say the same before the Snowden revelations if someone said NSA has tabs on pretty much everything...

Also, 99% of the AMD/Intel engineers won't know what they're putting in there. The "chip on chip" is ARM and IIRC in case of AMD is not even "theirs". It's ludicrous really. Very easy to have backdoors forced in like this. Just piggy-bag it in and make sure the "right people" enforce it under some nice cover story e.g. "management engine". You only need to keep them quiet with a FISC court order and viola.

[aht0]

I am afraid in 50 years you would be looking back today as "Golden Age of Privacy" regardless of what NSA, FSB/SVR or Chinese Intelligence Directorates are putting or not in their software and chips. Globalization, religious extremism (and terrorism it causes) and various other pressures are pretty much going to guarantee that situation with privacy goes relentlessly downhill. There is just no other way, actual risks brought by ever-quickening technical development are simply getting too big for allowing freedoms for the sake of freedoms themselves.

[pal666]

It's not about AMD.

i replied to "not all x86 processors support pcid". because amd x86 processors do not have to support pcid because they do not have bug to fix

[F.Ultra]

I wonder if you'd say the same before the Snowden revelations if someone said NSA has tabs on pretty much everything...

Also, 99% of the AMD/Intel engineers won't know what they're putting in there. The "chip on chip" is ARM and IIRC in case of AMD is not even "theirs". It's ludicrous really. Very easy to have backdoors forced in like this. Just piggy-bag it in and make sure the "right people" enforce it under some nice cover story e.g. "management engine". You only need to keep them quiet with a FISC court order and viola.

How exactly do you enforce a FISC court order on non US companies such as ARM and all the Chinese/Korean/Japanse manufacturers? Regarding NSA and Snowden, yes not only would I say that but I did back in the day. What we didn't know was to which extent NSA actually went (like intercepting Cisco hardware in shipping to add hardware backdors, I mean there where always speculation that such things could be done [I mean we knew that it happened to Xerox machines sold to the Soviets] but few actually believed that they would do it for machines sold to the US and Western Europe.

Further what we have gotten from the NSA and CIA leaks are that they actually do mid-shipment intercepts instead of i.e asking/telling Cisco to install said backdoors, probably because they know that they cannot trust such companies to keep silent. All tools that we have seen are also exploiting bought and discovered exploits and not sophisticated hidden backdoors.

And the 99% of engineers does not compute, implementing something complex as Out-of-Order Execution and Speculative Execution is not a small team. From design to construction that design have been combed over with literally microscopes by thousands of engineers over multiple of years. Intel employs over 100k people for a reason.

[F.Ultra]

i replied to "not all x86 processors support pcid". because amd x86 processors do not have to support pcid because they do not have bug to fix

The PCID instruction is not there to fix a bug, what happened was that Intel proposed that utilizing PCID would increase performance of the fix. The PCID instruction have been in Westmere since 2010 and was thus not added to fix a bug found in late 2017.

[Almindor]

How exactly do you enforce a FISC court order on non US companies such as ARM and all the Chinese/Korean/Japanse manufacturers?

Economic pressure? The same way the Empire of USA is enforcing everything these days. If they disagree it can go all political. Only contender could be China but I think USA still has too much power for them to blatantly risk it.

[pal666]

The PCID instruction is not there to fix a bug, what happened was that Intel proposed that utilizing PCID would increase performance of the fix.

eat some sugar and think again. the fix is to fix bug and pcid is to make fix less slow. cpus without bug do not need fix and do not need pcid