UKSM Is Still Around For Data Deduplication Of The Linux Kernel

[boxie]

DO NOT fuck with databases in any way.

Wuss

Just saying it could be a useful tool (and this is of course without trying it to see what the impacts or usefulness of it are)

[starshipeleven]

Wuss

Last guy that crashed a database server in a place I worked in (replaced the wrong RAID drive after a failure, obvious total loss and someone had to reload the thing from tape backups, took a day) risked his own life when talking to the boss, and lost a sizeable amount of money to pay damages. And also got fired afterwards.
Just saying.

Just saying it could be a useful tool (and this is of course without trying it to see what the impacts or usefulness of it are)

I'm just pointing out that you need some very compelling evidence to enable something like that as it carries a significant risk.

As said above, you can patch a linux kernel with this feature and do some tests with your favourite applications or databases and to see if it is worth it, but I suspect that if none of the serious sluggers (Google, RedHat, assorted computing companies, Intel, Oracle or other database makers) didn't give a fuck in nearly a decade, probably it's not that relevant overall or they are doing this already inside their own application.

Remember that Linux's bread and butter are either crappy embedded where RAM is at a premium (less so in more recent years), and various server or HPC usage where yes having all possible performance is good.

This isn't like enabling hardware acceleration for media in browsers... because who uses Linux for that anyway (obvious citation).

[boxie]

Last guy that crashed a database server in a place I worked in (replaced the wrong RAID drive after a failure, obvious total loss and someone had to reload the thing from tape backups, took a day) risked his own life when talking to the boss, and lost a sizeable amount of money to pay damages. And also got fired afterwards.
Just saying.

That sounds like quite the over reaction and would probably run afoul of employment laws (at least downunder). It is (at the very least) a carton offence though!
Still - backups were tested!

I'm just pointing out that you need some very compelling evidence to enable something like that as it carries a significant risk.

As said above, you can patch a linux kernel with this feature and do some tests with your favourite applications or databases and to see if it is worth it, but I suspect that if none of the serious sluggers (Google, RedHat, assorted computing companies, Intel, Oracle or other database makers) didn't give a fuck in nearly a decade, probably it's not that relevant overall or they are doing this already inside their own application.

Remember that Linux's bread and butter are either crappy embedded where RAM is at a premium (less so in more recent years), and various server or HPC usage where yes having all possible performance is good.

This isn't like enabling hardware acceleration for media in browsers... because who uses Linux for that anyway (obvious citation).

This does seem to be a performance boosting feature (the same way that having compression enabled on your file system is) and I am sure there would be some very interesting ways to exploit (and also mitigate against) it.

and while yes there is risk the same could be said for many features and applications that we currently use (web browsers are a great exploit vector) but we still use them

[starshipeleven]

This does seem to be a performance boosting feature (the same way that having compression enabled on your file system is) and I am sure there would be some very interesting ways to exploit (and also mitigate against) it.

and while yes there is risk the same could be said for many features and applications that we currently use (web browsers are a great exploit vector) but we still use them

As I said, there is currently no real proof that it provides big enough benefits to offset some pretty obvious attack vectors.
Browsers do usually provide big enough benefits to be worth the risk.

[xianai]

As I said, there is currently no real proof that it provides big enough benefits to offset some pretty obvious attack vectors.
Browsers do usually provide big enough benefits to be worth the risk.

Hi, I am the original author of the UKSM patch and I am still active. Recently I released a new bug-fixing version you can download at http://kerneldedup.org/projects/uksm/download/ .

I happen to have a PHD degree in CS majoring in information security. As far as I know, in any resource sharing systems, side/covert channels can be considered as everywhere. The only difference is how hard you can really exploit them and do real world harm. Stably expoliting a covert channel is VERY HARD compared to other attacks(e.g. SQL injection, buffer overflows, etc...). I believe there is no stable real world attacks(except few academic papers) to memory deduplication systems uptil now. UKSM has an advanced and complex algorithm, adding noises to the side/covert channels which makes them even harder to be exploited.

So, please, just stop talking theoretically and show me some code that can do real world damage, then I can craft counter measures.

[spangry]

Hi xianai, thanks for dropping in. Those benchmarks are pretty impressive. Just got a few question if you have some spare time:

- Does this just purely do dedupe, or is there also a compression step?
- Does UKSM play nice with zswap (which, I think, intercepts calls to frontswap)?
- You mention a custom hashing algo. It isn't this by any chance (https://cyan4973.github.io/xxHash/)?

Good luck with the project!

[xianai]

Hi xianai, thanks for dropping in. Those benchmarks are pretty impressive. Just got a few question if you have some spare time:

- Does this just purely do dedupe, or is there also a compression step?
- Does UKSM play nice with zswap (which, I think, intercepts calls to frontswap)?
- You mention a custom hashing algo. It isn't this by any chance (https://cyan4973.github.io/xxHash/)?

Good luck with the project!

@spangry, Thanks,

- No compression step yet.
- Currently both KSM and UKSM has no special consideration for the interaction with frontswap. However, chances are since UKSM is pretty fast, in many cases it will have already done its job before zswap starts. Of course, it's good to let UKSM and zswap fully work together someday.
- No, it's not xxHash. The hash algo has more interesting features than conventional hash algorithms. The purpose is not speed or avalanching effect alone, it's crafted to suit the scan process of UKSM.

[starshipeleven]

I believe there is no stable real world attacks(except few academic papers) to memory deduplication systems uptil now. UKSM has an advanced and complex algorithm, adding noises to the side/covert channels which makes them even harder to be exploited.

With all due respect as I'm not in your field so I cannot judge myself, but to actually think it's secure I need multiple credible sources, not just your word.

Having someone from grsecurity come out and say something about it would be a start.

[guglovich]

Where did it all go?