OpenZFS 2.2.2 & OpenZFS 2.1.14 Released To Fix Data Corruption Issue

[phoronix]

Phoronix: OpenZFS 2.2.2 & OpenZFS 2.1.14 Released To Fix Data Corruption Issue

Following a rare but nasty data corruption issue, OpenZFS 2.2.2 and OpenZFS 2.1.14 were released this evening to address the problem...

https://www.phoronix.com/news/OpenZFS-2.2.2-Released

[johncall]

Well now I'm curious about the change to cp in coreutils...

[Kjell]

Well now I'm curious about the change to cp in coreutils...

We need a ELI5

[IntrusionCM]

https://gist.github.com/rincebrain/e23b4a39aba3fadc04db18574d30dc73

https://github.com/openzfs/zfs/issues/15526#issuecomment-1826412289

https://github.com/openzfs/zfs/issues/15526#issuecomment-1826476737

"coreutils before 9.x did not support SEEK_HOLE/SEEK_DATA (it used a different method, FIEMAP, to detect holes, which ZFS does not support). In 9.x the code you're looking for is in lseek_copy() in copy.c."

[stormcrow]

For all the nay sayers about "this should have been discovered..." or "where was all the testing..." The bug could NOT be triggered before coreutils changed the way it does file copies. The likelihood of anyone discovering it without someone knowing exactly what they were looking for was slim to none like any other bug that would be impossible to find when the code path to reach it doesn't exist.

[Kjell]

For all the nay sayers about "this should have been discovered..." or "where was all the testing..." The bug could NOT be triggered before coreutils changed the way it does file copies. The likelihood of anyone discovering it without someone knowing exactly what they were looking for was slim to none like any other bug that would be impossible to find when the code path to reach it doesn't exist.

Unhandled events shouldn't be able to corrupt data though..

[stormcrow]

Unhandled events shouldn't be able to corrupt data though..

And you'll never find it without having a code path to the bug or a deliberate systematic logic diagram that includes all possible paths. "Unhandled events" like this one shouldn't be reached by definition so no amount of testing on a system that never reaches those paths can find them. You have to do a deliberate code search and mapping to find them. Which basically means there should be no unhandled events because all events lead to a properly handled logic block. Either way, this bug wouldn't be discoverable without a systematic logic mapping, not by endless testing of paths that couldn't be reached to begin with.

[justinkb]

For all the nay sayers about "this should have been discovered..." or "where was all the testing..." The bug could NOT be triggered before coreutils changed the way it does file copies. The likelihood of anyone discovering it without someone knowing exactly what they were looking for was slim to none like any other bug that would be impossible to find when the code path to reach it doesn't exist.

The zfs logic was always flawed (apparently since 2006 even perhaps). The coreutils change only exposed the flaw. Your assertion it could not be triggered before coreutils change is just absurd. There is so much software out there, you have no idea what other software could already have been having the same effect coreutils 9.2+ does. The fact that it corrupted data silently is particularly bad for diagnosing whether you've been hit by this. You'd need to check all your data for holes that shouldn't be there (very unlikely to be a viable method) or use known good checksums that aren't from the filesystem itself (unlikely to exist for a lot of the data in question)

[Volta]

File system made to prevent silent data corruption 'introduced' silent data corruption. What an irony.

[Rallos Zek]

File system made to prevent silent data corruption 'introduced' silent data corruption. What an irony.

ZFS fails again!

ZFS has been proven pretty flakey and unreliable for many years.