Optane SSD RAID Performance With ZFS On Linux, EXT4, XFS, Btrfs, F2FS

[F.Ultra]

Which was my point, the "overhead" is what you see there, the rest is IO and buses.

For raid1, you DO get a better read performance. For raid0 you DO get a better write performance etc etc

The added latency for the higher modes is all the writes.

The actual algorithm it self is nothing compared to just leaving the cpu, the point is that the "overhead" is small.

No, what you see there is the "overhead" of just the raid algorithm part of md when it tries to raid a single block chunk that is already hot in the cache. That algorithm is a very small subset of the code path of md (so it's not just add IO and buses but also lots and lots of code running CPU cycles) while also you will never ever in the real world be performing raid of a single block over and over from hot cache.

Again, these are not performance numbers, they are just there to determine which algorithm to use inside md. They have no usage outside that decision.

[jrch2k8]

+1
Think the above would make ZFS competitive in the throughput benchmarks? I'm planning to use two SSDs in raidz to prevent dataloss in the case that one dies. Nice to see the latency is low. I'm planning to be running multiple VMs on a root ZFS system.

well ZFS is not a speed winner filesystem but a very damn trustworthy one but yes with some optimization it can fare way better than those default results.

Please if you have 2 SSD don't use RAIDZ since those are meant for parity raids, aka RAID3,5,6 instead use mirror aka RAID1.

For virtualization i would recommend KVM but google around which options fit best for your virtualization strategy.

If you are using linux(in some guest OS cases) is better to use systemd containers within a ZFS volume

Don't enable DEDUP on any of the KVM virtualization and if you enable it on the containers make sure you have the ram to back it up, it can be pretty intensive on resources.

[deusexmachina]

Please if you have 2 SSD don't use RAIDZ since those are meant for parity raids, aka RAID3,5,6 instead use mirror aka RAID1.

Why? What if you want to use SSDs, ZFS, and you want one of your SSDs to be able to die without losing data? Then you need raidz, right?

[kmand]

A more conventional ZFS use of the octane, at least on a larger server would be for the ZIL not the actual disk storage. That takes advantage of octane's NV, but may be is overkill especially in size.

[deusexmachina]

A more conventional ZFS use of the octane, at least on a larger server would be for the ZIL not the actual disk storage

I can't parse this. ZFS on octane would be used only for the ZFS Intent Log? Meaning the ZFS Intent Log would just be stored on an octane disc and the rest on HDDs or regular SSDs? Why?

[kmand]

I can't parse this. ZFS on octane would be used only for the ZFS Intent Log? Meaning the ZFS Intent Log would just be stored on an octane disc and the rest on HDDs or regular SSDs? Why?

On large ZFS servers (especially NFS servers) the data is going to be on HDD's. Many TBs of data are just too expensive for SSDs.

The ZIL is written with synchronous writes and is required to be on non-volatile storage. Optane is a lot faster than conventional SSD and is non-volatile. Optimally you want the ZIL on its own device so as not
to have to compete with the data disks.

My comment on overkill is that the ZIL does not really need to be huge, and may be a waste given the size of Optane.

[jrch2k8]

Why? What if you want to use SSDs, ZFS, and you want one of your SSDs to be able to die without losing data? Then you need raidz, right?

zpool create tank drive..A drive..N
this command equals STRIPE, this mean raid0 aka each disk have a portion of data but not all, extremely insecure

zpool create tank mirror drive..A drive..N(in some OSes and ZFS versions mirror is similar enough to RAIDZ(without number) but mirror is recommended on Zol)
this command equals Mirror, this mean raid1 aka each drive have 1 full copy of the data, relatively secure(<- this is what you are asking for)

zpool create tank raidzN drive..A drive..N
this command equals PARITY, this mean raid5/6/7 aka data is distributed among the data disk while the parity and location are on a separated disk knows as parity drive, see that the N in raidzN means the amount of parity drives, from secure to very secure

[deusexmachina]

zpool create tank drive..A drive..N
this command equals STRIPE, this mean raid0 aka each disk have a portion of data but not all, extremely insecure

zpool create tank mirror drive..A drive..N(in some OSes and ZFS versions mirror is similar enough to RAIDZ(without number) but mirror is recommended on Zol)
this command equals Mirror, this mean raid1 aka each drive have 1 full copy of the data, relatively secure(<- this is what you are asking for)

zpool create tank raidzN drive..A drive..N
this command equals PARITY, this mean raid5/6/7 aka data is distributed among the data disk while the parity and location are on a separated disk knows as parity drive, see that the N in raidzN means the amount of parity drives, from secure to very secure

That's my understanding, but you said: "Please if you have 2 SSD don't use RAIDZ since those are meant for parity raids" -- So I was wondering what you're saying to do (rather than not to do).

[pal666]

Note, pcie and memory will be an issue WAY before there is an issue with raid.

note, it wasn't me who argued that "Optane is already so fast, any extra complexity you add can make it worse"

[GreenReaper]

Read-write performance on BTRFS will suffer in pgbench and MariaDB without something like "chattr +C '$datadir' 2>/dev/null" before you create the clusters. Looking at the test scripts, it's not clear that this is being done. (Debian will if its pg_createcluster script is used.)