Fedora Stakeholders Debate Statically Linking Python For Better Performance

[starshipeleven]

i'm pretty sure productive and scientific python is thin wrapper around real implementation in faster language, i.e. it will not benefit from subj at all

You are pretty wrong. There is plenty of small and medium research projects done with Python. Same story as with Java (Matlab), the researchers aren't true programmers.

[pal666]

Bash is not used for actual productive software and scientific work (not as the main payload logic anyway)

i'm pretty sure productive and scientific python is thin wrapper around real implementation in faster language, i.e. it will not benefit from subj at all

[oleid]

And that's the complicating factor. The controversy of the proposal isn't really over disk space, it's over the consequences of the change throughout the system. Nobody really cares if Python itself statically links to it's own libraries... but there's a _lot_ of stuff in a modern distro which links to libpython to provide scripting functionality. Lots of risk, balanced by a benchmark performance gain that may or may not benefit real usage.

Maybe the interpreter should be statistically linked and the rest use a dynamic version of libpython?

[Delgarde]

There are other programs, which use libpython, i.e. some $WINDOW_MANAGER, which uses it for internal scripting stuff. They will possibly get statically linked, too.

And that's the complicating factor. The controversy of the proposal isn't really over disk space, it's over the consequences of the change throughout the system. Nobody really cares if Python itself statically links to it's own libraries... but there's a _lot_ of stuff in a modern distro which links to libpython to provide scripting functionality. Lots of risk, balanced by a benchmark performance gain that may or may not benefit real usage.

[ermo]

Ah yes, I was sure someone will imply that if redundancy is bad on Windows, it must be because Windows suxx, therefore Windows developers are idiots. And Linux is awesome, therefore Linux developers are geniuses, therefore, the lessons learned on Windows are somehow irrelevant and Linux apps are somehow immune to the inherent problems of static linking.

If you still can't tell if that was sarcasm or not, let me help you out: GDI and msvcrt.dll were just metaphores and examples of the underlying problem.

Regarding Go, I haven't really looked into it, if they do that, I retract my statement about that.

All I said was that, since Go appears to do the things you warn against, how is Go a good example compared to statically linking Python (which you appear to warn against by specifically referencing a specific MS GDI+ example)?

I also never implied that Windows sucks nor that Windows developers are idiots etc. That part is all on you. I thought it obvious that the software development part of the IT community as such should learn from the bad old Windows days of bundling DLLs willy nilly without bothering to update them even after serious security vulnerabilities become known.

The only thing 'the bad old days of MS software development' was supposed to refer to is the fact that software distribution on Windows (outside of MS controlled code bases) never really had the kind of central repositories that the Linux community now takes for granted (then again, 2004 was 15 years ago. Internet access has become much more prevalent since then). This has traditionally made it impossible to reap the benefits of dynamic linking on a repository-wide basis for Windows developers.

These days, the closest equivalent to a central Linux distribution repository for Windows is probably chocolatey (which is maintained by a 3rd party)?

[profoundWHALE]

There is nothing in the proposal from Fedora that will make python less safe.

I'm not saying that the change would make them suddenly full of security holes. The change brings along some additional security concerns that have to do with versions of python.

[F.Ultra]

Imagine having a program language become such a bottleneck that you decide to make it less safe to gain a few percentage points.

At this point you might as well leave it as is and slowly work on replacing those vital program bits with something that is hundreds of times faster.

There is nothing in the proposal from Fedora that will make python less safe.

[F.Ultra]

it doesn't work that way. for it someone has to globally decide which part of address space every library on machine takes, otherwise you'll have two libraries or library and executable clash. prelink tried to do that and prelink is obsolete. and that was even before address space layout randomization

Thanks, the VM is what I was forgetting. If we ignore ASLR however it should be possible to at least keep the same offsets even when the virtual memory changes for the new binary and then the calls could all use relative jumps which would make it work with a single pass at the first load.

[profoundWHALE]

Imagine having a program language become such a bottleneck that you decide to make it less safe to gain a few percentage points.

At this point you might as well leave it as is and slowly work on replacing those vital program bits with something that is hundreds of times faster.

[starshipeleven]

unless you are using lto/pgo like subj

Only a few performance-critical packages need that