Tweet
The replacement change/proposal to the previously discussed statically-linking Python in Fedora:
Cheers,
Mike
When we build the Python interpreter with the -fno-semantic-interposition compiler/linker flag, we can achieve a performance gain of 5% to 27% depending on the workload. Link time optimizations and profile guided optimizations also have a greater impact when python3 is built this way.
As a negative side effect, it disables the LD_PRELOAD feature: it's no longer possible to override symbols in libpython with LD_PRELOAD.
Interposition is enabled by default in compilers like GCC: function calls to a library goes through a "Procedure Linkage Table" (PLT). This indirection is required to allow a library loaded by LD_PRELOAD environment variable to override a function. The indirection puts more pressure on the CPU level 1 cache (instruction cache). In terms of performance, the main drawback is that function calls from a library to the same library cannot be inlined, to respect the interposition semantics. Inlining is usually a big win in terms of performance.
Disabling interposition for libpython removes the overhead on function calls by avoiding the PLT indirection, and allows to inline more function calls. We're describing function calls from libpython to libpython, something which is very common in Python: almost all function calls are calls from libpython to libpython.
If Fedora users need to use LD_PRELOAD to override symbols in libpython, the recommended way is to build a custom Python without -fno-semantic-interposition.
It is still possible to use LD_PRELOAD to override symbols in other libraries (for example in glibc).
As a negative side effect, it disables the LD_PRELOAD feature: it's no longer possible to override symbols in libpython with LD_PRELOAD.
Interposition is enabled by default in compilers like GCC: function calls to a library goes through a "Procedure Linkage Table" (PLT). This indirection is required to allow a library loaded by LD_PRELOAD environment variable to override a function. The indirection puts more pressure on the CPU level 1 cache (instruction cache). In terms of performance, the main drawback is that function calls from a library to the same library cannot be inlined, to respect the interposition semantics. Inlining is usually a big win in terms of performance.
Disabling interposition for libpython removes the overhead on function calls by avoiding the PLT indirection, and allows to inline more function calls. We're describing function calls from libpython to libpython, something which is very common in Python: almost all function calls are calls from libpython to libpython.
If Fedora users need to use LD_PRELOAD to override symbols in libpython, the recommended way is to build a custom Python without -fno-semantic-interposition.
It is still possible to use LD_PRELOAD to override symbols in other libraries (for example in glibc).
Cheers,
Mike