PostgreSQL Begins Working On Zstd Compression Support

[phoronix]

Phoronix: PostgreSQL Begins Working On Zstd Compression Support

While PostgreSQL has supported compression with its TOAST storage and over the past year has built-up LZ4 compression support for it along with compressing the WAL, backup compression, and other usage, PostgreSQL developers are preparing to further extend their compression capabiities with Zstd support...

https://www.phoronix.com/scan.php?page=news_item&px=PostgreSQL-ZStd-Start

[intelfx]

The unstoppable crusade of Zstandard continues.

About time! This algorithm really is the best thing that's happened to general purpose data compression since DEFLATE.

[Imroy]

Zstd all the things!

[cl333r]

The unstoppable crusade of Zstandard continues.

About time! This algorithm really is the best thing that's happened to general purpose data compression since DEFLATE.

Yeah, also the API is user friendly and well thought out, took me a short time to figure out how to use it.

Edit: I wish ODF files moved from zip to zstd or allowed it as an option.

[microcode]

I kinda don't see the point of merging a branch to build a library that is completely unused in the codebase; surely you should have at least one user of it before doing that.

[microcode]

Yeah, also the API is user friendly and well thought out, took me a short time to figure out how to use it.

Edit: I wish ODF files moved from zip to zstd or allowed it as an option.

Honestly, I feel like you'd be better off designing a clean sheet successor to ODF. My colleague and I built a toolchain that compiles a subset of ODS, and the ODS reader is by far the ugliest and most error-prone part of it (we had to do custom for a variety of reasons I won't go into). Particularly the way LibreOffice writes ODS files is appalling; whatever locale you're in, it'll introduce indic and han font selections to every style, it'll add a bunch of automatic styles into your file, and it'll rename all of them when you save, even if you don't change anything. Taken together, this all leads to a lack of useful change management and comparison tools.

Then there's the issue that a reasonably complex ODS is at least ten times less compact than it could otherwise be, if for no other reason than treating relative cell references as unique for each cell even when the offset is identical, and thus making row and column repeating useless with formulas.

[cl333r]

Honestly, I feel like you'd be better off designing a clean sheet successor to ODF. My colleague and I built a toolchain that compiles a subset of ODS, and the ODS reader is by far the ugliest and most error-prone part of it (we had to do custom for a variety of reasons I won't go into). Particularly the way LibreOffice writes ODS files is appalling; whatever locale you're in, it'll introduce indic and han font selections to every style, it'll add a bunch of automatic styles into your file, and it'll rename all of them when you save, even if you don't change anything. Taken together, this all leads to a lack of useful change management and comparison tools.

Then there's the issue that a reasonably complex ODS is at least ten times less compact than it could otherwise be, if for no other reason than treating relative cell references as unique for each cell even when the offset is identical, and thus making row and column repeating useless with formulas.

I've been wondering why they include those asian styles into every .ods document. Anyway, a while ago I created a library to work with .ods files including formulas [1], but since the ods spec is 1k pages I implemented the most useful stuff, enough to create an invoice and such.


[1] https://github.com/f35f22fan/Ods2

[microcode]

I've been wondering why they include those asian styles into every .ods document. Anyway, a while ago I created a library to work with .ods files including formulas [1], but since the ods spec is 1k pages I implemented the most useful stuff, enough to create an invoice and such.

[1] https://github.com/f35f22fan/Ods2

Cool, I'm kinda mulling over my own take on spreadsheets too. I've thought about having first-class iteration, including nested multi-cell iteration and indefinite sequences.

[cl333r]

Cool, I'm kinda mulling over my own take on spreadsheets too. I've thought about having first-class iteration, including nested multi-cell iteration and indefinite sequences.

The library does support having nested cells/tables, which is why I named a cell method GetFirstString() [1] because a cell can have varying objects inside which in turn might contain random objects. Though doing it from the official API IIRC isn't documented because I imagined barely anyone would need it.

[1] https://github.com/f35f22fan/Ods2/bl...r/ods/Cell.hpp

[GreenReaper]

This is great, but what'd be even greater is getting zstd compression of streaming replication working, aka libpq compression.

Right now I have to patch OpenSSL in two places and recompile it to restore the SSL compression feature, the second time to subvert an attempt to outright disable it, having stripped out support without a replacement.

I appreciate this is not a feature useful to many, but it is to those of us on limited bandwidth or transfer budgets.