Fedora Stakeholders Debate Statically Linking Python For Better Performance

[starshipeleven]

Disk space use to be very cheap until SSDs came along ...

It's improving rapidly.

[starshipeleven]

I did not say to use static linking. I actually don't support that very much as each library security update requires rebuilding or relinking all the executables. I mean to think about something more innovative.

rebuilding/relinking isn't a huge issue with ccache

[Raka555]

instead of having the shared object mapped once in each process using it, you would have a copy-on-write modified version in each process for the relocated jumps all over the place.

There won't be copy-on-write versions of the library in memory of each process.
The code that was linked from the library will be heavily optimized and integrated into the executable at a fraction of the size it were in the library.

[rene]

rebuilding/relinking isn't a huge issue with ccache

Well, I for one don't want GB of ccache on my system, plus if you rebuild more stuff usually some header changed a little bit here and there causing large scale cache invalidation anyway, ...

[Raka555]

The biggest stumbling block for static linking is actually the GPL license ...

I wish they would make an exception for static linking as part of the default license.

[rene]

There won't be copy-on-write versions of the library in memory of each process.
The code that was linked from the library will be heavily optimized and integrated into the executable at a fraction of the size it were in the library.

I thought this thread was talking about shared objects, it sound more like you talk about static libraries. Also "fraction of the size it where in the library" sound a bit like a myth, as most libraries have plenty of internal dependencies, like not causing that large "savings" for a non-trivial more than "hello world" applications. Despite each, even fractional copy of a library inside each and every executable is space wasted over the shared object approach invented (among other things) exactly for this reason, ...

[starshipeleven]

Well, I for one don't want GB of ccache on my system, plus if you rebuild more stuff usually some header changed a little bit here and there causing large scale cache invalidation anyway, ...

Are you rebuilding your distro from your personal PC?

I don't even have my own distro but I have a dedicated "buildserver" for my OpenWrt stuff (an old shit PC with a bunch of old 512GB hard drives in RAID0), and with ccache it's pretty decent.

[starshipeleven]

Despite each, even fractional copy of a library inside each and every executable is space wasted over the shared object approach invented (among other things) exactly for this reason, ...

It's always a tradeoff between things. It's just that the "wasted disk and RAM space" weights less now in (near)2020 than back in the day.

[rene]

It's always a tradeoff between things. It's just that the "wasted disk and RAM space" weights less now in (near)2020 than back in the day.

Well, I also find recompiling all the applications and libraries on each daily security issue( https://www.youtube.com/watch?v=vPlzP_aQB3Y) a bit time, CPU cycle and energy consuming. For that alone I would still prefer something like shared objects.

[rene]

Are you rebuilding your distro from your personal PC?

I don't even have my own distro but I have a dedicated "buildserver" for my OpenWrt stuff (an old shit PC with a bunch of old 512GB hard drives in RAID0), and with ccache it's pretty decent.

yes, I have my own distribution (https://t2sde.org), and the last time I used ccache was a decade ago, as testing all the various compiler updates, variants (gcc vs. clang) and c libraries (glib, must) instantly invalidate all the ccache anyway. And even if you only build amd65/glibc/gcc then each minor update here and there invalidates most of it anyway. Not to mention hat some new stuff (e.g. rust) does not even cache very well yet (or at all). I have a 64GB RAM Ryzen 2xxx, waiting for the 3950X for the next compile speedup: https://www.youtube.com/watch?v=1mkf0O-f4hU