Germany's 1&1 Still Working On MARS For The Linux Kernel, Still Hoping For Upstream

[phoronix]

Phoronix: Germany's 1&1 Still Working On MARS For The Linux Kernel, Still Hoping For Upstream

At the end of last year was an update on MARS Replication System Still Being Worked On For Upstream Linux Kernel and like clock work, the German web hosting provider has issued another update on the in-development MARS replication system and is still hoping to mainline it, maybe next year...

http://www.phoronix.com/scan.php?page=news_item&px=MARS-2016-Linux-Kernel

[Chewi]

Must have missed this one last time around. Sounds like an interesting competitor to Gluster's geo-replication. We're using the latter at work and plan to use it more soon. It works well but I don't think it's as fancy as they make it out to be. It's basically rsync on steroids.

[DrYak]

@Chevi:
MARS is a derivative of the DRDB that is currently already in the kernel.
They differ from Gluster geo-replication with that Gluster is a cluster file system (an FS specially designed with computer clusters in mind).

Whereas MARS and DRDB are only replicating raw block devices
and it's up to you to put any file system of your choosing above, e.g.: EXT4.
(think of the failover/high availability offered by RAID1, but synced over a network connection instead of accross 2 disks on the same machine).
If server A goes down, server B has the same content on its disks as server A right up to the point where the crash occured)

Or you can choose to elect a special file system that is designed to be accessible by multiple machines at the same time
(some of the cluster oriented FS can do this, usually relying on a lock manager. You could in theory mount the same block device from the same SAN on 2 different cluster nodes. Now you do the same but instead there is no SAN, the nodes are keeping their local block devices in sync over the network).

The minor difference between the original DRBD and MARS, is that DRBD makes no guarantee about the order in which the sync happens.
Blocks that are mis-matched will simply get synched eventually.
Whereas MARS ensures that the sync will happen in chronological order.

Thus, if a failure happens mid-sync, with DRBD the data might end-up inconsistent with blocks comming from different point in time, some already synced, other not.
If a failure happens mid-sync, with MARS, the data will correspond to what the source server had in the past. All the block until instant T are synced, all the blocks after that instant will be missing.

Over a slow connection (like the geo replication that 1&1 is doing this this - as opposed of using an infiniband interconnect), when there's a high risk that sync won't be finished, it increases the chance that the half replicated block device can be read.
(Specially if coupled with a CoW or log structured file system like BTRFS, ZFS, F2FS, etc. - these FS always make sense at any point in time)


I haven't had experience with MARS, but we did some tests with DRBD over Infiniband in one of the universities I've worked in.