PostgreSQL Optimizes Performance & Lower Memory Management Overhead
Postgres' David Rowley landed a rather significant change to improve the performance of and reduce overhead of memory management for this database server. The change reduces the size of the chunk header size from 16 to 8 bytes and in turn this work should lead to much more efficient memory use.
The changes being made here reduce the chunk header size down to just 8 bytes for all 3 of our memory context types. For small to medium sized allocations, this significantly increases the number of chunks that we can fit on a given block which results in much more efficient use of memory.
Additionally, this commit completely changes the rule that pointers to palloc'd memory must be directly prefixed by a pointer to the owning memory context and instead, we now insist that they're directly prefixed by an 8-byte value where the least significant 3-bits are set to a value to indicate which type of memory context the pointer belongs to. Using those 3 bits as an index (known as MemoryContextMethodID) to a new array which stores the methods for each memory context type, we're now able to pass the pointer given to functions such as pfree() and repalloc() to the function specific to that context implementation to allow them to devise their own methods of finding the memory context which owns the given allocated chunk of memory.
...
More details on this PostgreSQL performance optimization around memory overhead can be found via this commit.
An earlier discussion around reducing the PostgreSQL memory overhead saw Rowley enjoying a 17% performance increase in throughput for a read-only simple workload on a Ryzen Threadripper workstation.
It will certainly be interesting to see how this memory overhead reduction helps out in the real-world when it debuts in a stable PostgreSQL release. But with PostgreSQL 15 already being branched ahead of its release later this year, this change isn't expected until PostgreSQL 16.
10 Comments
