The Cost of SELinux, Audit, & Kernel Debugging

[papo]

Improve graphs

The 11th Commandment: Thou shalt not create a graph without reading and understanding "The Visual Display of Quantitative Information" by Edward Tufte.

I would like to second this opinion. Please improve the graphs. I've been reading the benchmark reports on phoronix for quite some time and I understand that a lot of time and efforts are spent on these test cases, but the way the results are presented is just inacceptable and unusable for those people who really want to interpret the graphs. This has been mentioned over and over and over again in the discussions of the articles, but for reasons I fail to understand, has not been implemented even though a load of improvement suggestions have been provided in the past. This is even worse in the context that a new version of the test suite was released recently and that other reviewers are being encouraged to use it. I know this is not science here but please alter the graphs such that they reflect the quality of the test of the article.

[ThePowerTool]

I don't see any documentation on the modifications you made. Just the descriptions of what your modifications were supposed to disable. I would question the validity of any test that doesn't properly document the setup, configurations, and preparation.

Did I miss it?

Thanks!

[dlang]

This is actually imo a much more important test than testing no protection against SELinux. I mean, every former Windows user knows too that a Windows runs *much* faster without any antivirus programs running too. But benchmarking security solutions against each other sounds much more useful. Might in fact even be worth the effort of creating two Gentoo installations on identical machines, one with SELinux and one with AppArmor. The systems should be otherwise identical. Then see how they fare.

if you try to benchmark every protection method against each of the other ones you end up with a huge number of tests to run (and you need to be expert at all of the methods to configure them as efficiantly as possible for a fair comparison)

if instead you do comparisons against a baseline of a bare system, it's easier to have different groups do different comparisons and compare the results.

AppArmor doesn't try to do all of the same protections that SELinux tries to do, and that needs to be kept in mind when doing the comparisons.

it would also be interesting to see each of these tools with the distro-default configurations vs a 'permit anything' configuration (on the basis that the 'permit anything' is about the minimum overhead you can get while still having the tool configured)

[dlang]

The cost of cleaning up after a breakin will totally nullify any "savings" from getting rid of SELinux. Google for "SELinux apache vulnerability".

"Belt-and-suspenders" is how the rest of the engineering world works. Bridges and buildings are overbuilt by a factor of 3. Divers carry extra air. Two-engine planes can fly with one engine. These engineers have managed to come to grips with the fact that they are not perfect. Only in software do you see such reliance on a single level of protection.

SELinux is far from the only way to defend a server. it is _a_ way, and there are times when it is appropriate.

SELinux may or may not reduce the cost of cleaning up after a break-in. if the SELinux policies are tight enough they _may_ prevent a break-in, but if they don't completely prevent it, there can still be a lot of cleanup to do. Also, if you have good server build automation tools, cleaning up the server can be a matter of rebuilding it in a few min (and in either case you need to analyze how the break-in happened and update stuff to prevent it from happening again, running known-broken stuff and counting on SELinux to prevent the exploits against that software from damaging you too badly is not a very good position to take)

[nanonyme]

It would also be interesting to see how much AppArmor hits system performance as well.

This is actually imo a much more important test than testing no protection against SELinux. I mean, every former Windows user knows too that a Windows runs *much* faster without any antivirus programs running too. But benchmarking security solutions against each other sounds much more useful. Might in fact even be worth the effort of creating two Gentoo installations on identical machines, one with SELinux and one with AppArmor. The systems should be otherwise identical. Then see how they fare.

[deanjo]

It would also be interesting to see how much AppArmor hits system performance as well.

[frantaylor]

Actually isn't the major reason why Fedora has SELinux that RHEL wants SELinux because their enterprise server consumers have an appreciation for system security and have plenty of money to buy a fast server?
But yeah, it's possible a generic end-user might want to think it over about SELinux and such factors before choosing a distro.

The cost of cleaning up after a breakin will totally nullify any "savings" from getting rid of SELinux. Google for "SELinux apache vulnerability".

"Belt-and-suspenders" is how the rest of the engineering world works. Bridges and buildings are overbuilt by a factor of 3. Divers carry extra air. Two-engine planes can fly with one engine. These engineers have managed to come to grips with the fact that they are not perfect. Only in software do you see such reliance on a single level of protection.

[nanonyme]

The Apache static page serving test shows Fedora to be about 10x slower than Mandriva/SUSE/Ubuntu. So it's not exactly worthless, IF this is related to SELinux

Also, it probably is more of an argument to avoid a distro with SELinux, than it is an argument to recompile that distro

Actually isn't the major reason why Fedora has SELinux that RHEL wants SELinux because their enterprise server consumers have an appreciation for system security and have plenty of money to buy a fast server?
But yeah, it's possible a generic end-user might want to think it over about SELinux and such factors before choosing a distro.

[yesterday]

When you consider the pain and time involved, you'd have to use it for years before it paid off. Look at those graphs, we are talking about a couple of percent. How much is your time worth? Even on a server you would be better off investing in faster hardware to overcome the performance difference.

The Apache static page serving test shows Fedora to be about 10x slower than Mandriva/SUSE/Ubuntu. So it's not exactly worthless, IF this is related to SELinux

Also, it probably is more of an argument to avoid a distro with SELinux, than it is an argument to recompile that distro

[yesterday]

What would you recommend ?

I don't want to condone a pugnacious attitude, but surely the way would be ask suggested above. E.g. compiling kernel and world with and without SElinux.



Anyway Does this explain Fedora's poor Apache static webserving perfomance? Are the other distros shipping SELinux not integrating it as much as Fedora? Even without SELinux enabled in the kernel, it's about 10x slower (as seen in the recent Ubuntu/SUSE/Mandriva/Fedora shootout)