Linux DM-VDO "Virtual Data Optimizer" Preparing To Land In The Upstream Kernel
The DM-VDO code has made it into DeviceMapper's "for-next" branch meaning that it should soon make it into the mainline kernel. Though as of writing it hasn't yet been added to the "dm-6.9" branch that also exists in the DeviceMapper Git tree so we'll see if it gets held off until Linux v6.10 or still makes it for the v6.9 merge window coming in the next week or two.
The DM-VDO documentation describes the Virtual Data Optimizer as:
The dm-vdo (virtual data optimizer) target provides inline deduplication, compression, zero-block elimination, and thin provisioning. A dm-vdo target can be backed by up to 256TB of storage, and can present a logical size of up to 4PB. This target was originally developed at Permabit Technology Corp. starting in 2009. It was first released in 2013 and has been used in production environments ever since. It was made open-source in 2017 after Permabit was acquired by Red Hat. This document describes the design of dm-vdo. For usage, see vdo.rst in the same directory as this file.
Because deduplication rates fall drastically as the block size increases, a vdo target has a maximum block size of 4K. However, it can achieve deduplication rates of 254:1, i.e. up to 254 copies of a given 4K block can reference a single 4K of actual storage. It can achieve compression rates of 14:1. All zero blocks consume no storage at all.
Via this Git merge brings DM-VDO into the DeviceMapper for-next branch.
Red Hat engineers have been working on bringing the DM-VDO target to the mainline kernel for a while so it's nice to see this effort soon culminating. There is the VDO user-space tools on GitHub for managing pools of deduplicated and/or compressed storage.
THis is a big addition with the DM-VDO merge coming in at 53.4k lines for the kernel code and documentation.
22 Comments
