Fast Dedup Coming To OpenZFS For Overhauling Deduplication Capability

The folks at iXsystems and Klara are contributing Fast Dedup support to upstream OpenZFS and beginning to roll out this improved deduplication support within TrueNAS SCALE starting next month.

OpenZFS Fast Dedup overhauls the file-system's deduplication capability to address the need of deduplication hash tables being in memory at all times and other performance/scalability challenges. The new OpenZFS Fast Dedup will now have the size of the metadata be dynamically sized to fit in RAM or dedicated flash devices, the metadata structure has been re-engineered, the dedup table will favor dedup-able data and prune blocks that show no dedup potential, and various other improvements.

More details on the OpenZFS Fast Dedup feature can be found via the iXsystems blog.