DatArcs Is Aiming For Dynamically-Tuned, Self-Optimizing Linux Servers

[cj.wijtmans]

No, what I think it's doing is closer to Bayesian optimization of hyperparameters. A bit like Spearmint, but applied to Linux systems rather than some machine learning model.

You run an experiment with settings S[t]. You observe the outcome O[t]. Based on the outcome you tweak the settings a bit and start another time period with settings S[t+1]. Repeat. Eventually you can converge on a solution that's near-optimal according to the chosen criteria (e.g. throughput, latency, energy use, cost, ...)

So i assume you have to make profiles yourself or do they make them for you.

[rmoog]

They got nothing on Windows Optimizer

[sarfarazahmad]

rhel7 has a tuned daemon.

[Idonotexist]

So basically its doing what a scheduler should be doing just in a wider area? Could it not be contributed as module into the kernel?

No, what I think it's doing is closer to Bayesian optimization of hyperparameters. A bit like Spearmint, but applied to Linux systems rather than some machine learning model.

You run an experiment with settings S[t]. You observe the outcome O[t]. Based on the outcome you tweak the settings a bit and start another time period with settings S[t+1]. Repeat. Eventually you can converge on a solution that's near-optimal according to the chosen criteria (e.g. throughput, latency, energy use, cost, ...)

Ubuntu 16.40

5chars

[EmbraceUnity]

Relatedly, Google's DeepMind project was used to cut cooling costs via machine learning. This makes me hopeful that one might be able to use ML to optimize energy use and performance across the board, at many layers of the stack.

https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/

[starshipeleven]

Ah cool, we needed some "your pc is slow, install $random_bs" on Linux too. /sarcasm


More seriously, this is very interesting even if closed source, as the config changes are still readable and being automated it can do much more accurate testing than a human can.

[Michael]

So basically its doing what a scheduler should be doing just in a wider area? Could it not be contributed as module into the kernel?

Unlikely they would contribute it as a module to the kernel since it appears they are building a commercial organization around it. Additionally, it also looks like they do some application-level tweaking too, so isn't just all kernel tuning being done.

[cj.wijtmans]

So basically its doing what a scheduler should be doing just in a wider area? Could it not be contributed as module into the kernel?

[phoronix]

Phoronix: DatArcs Is Aiming For Dynamically-Tuned, Self-Optimizing Linux Servers

DatArcs is a new software start-up aiming to provide software to dynamically tune Linux servers for maximum performance and energy efficiency in the data-center. The DatArcs optimizer analyzes the server's workload over time and optimizes the server "several times per minute" to achieve better performance or lower power use...

http://www.phoronix.com/scan.php?page=news_item&px=DatArcs-Phoronix-Test