Another Look At The Bcachefs Performance on Linux 6.7

[KJ7LNW]

The introduction page says that the 64 core 7980x is running at 8.21GHz. That seems to be on the high side of things, 3.2GHz is more reasonable, typo?

Indeed! Is this liquid nitrogen cooled or something? Michael, is that number real? If so, where is the article about your 8GHz accomplishment?

[LinAdmin]

a) yes they do, they have e.g different sensors for vibration and what not that does not exist on normal consumer grade hdds, one of the reason why the MTBF is so much higher on these than on the consumer versions.

b) you do realise that this number indicates that they are fully aware that there is a chance that the same read does not return the exact same data every time? Now that is a very low chance (since again this is a server grade hdd) but it is still there and it is a number they have figured out what it is (since their other drivers have different numbers).

c) data stored on hdds are via magnetic fields so unplugging the drive for 10-20 years have a high chance of altering that magnetic field. This is very simple physics. Not sure why you think constantly having to rewriting the whole partition is somehow a better solution than simply adding checksums ala zfs, btrfs and bachefs, but hey you do you.

I have been driving a car for 31 years now and have never ever had a need for either a seatbelt nor an air-bag, going by your logic I should now rant on car forums that car crashes is a myth...

Ad
a) For all magnetic disks it is very important to avoid too much vibration because vibration disturbs the very small air gap between platter and head and therefore results in very bad reliability of recording and playback,

b) Of course the probability of an erroneous operation is very low for consumer and server grade disks. I would not rely on the numbers given because measuring it is very time consuming and costly. Just use self repairing Raid-1 and forget that kind of problem.

c) Your statement regarding an effect of unplugging is just plain wrong. Yes, it is Physics and I hold a degree in experimental physics :-D

I did not recommend "constantly having to rewriting the whole partition". But doing it once per a few years does bring the advantages I wrote.

Your very last sentence is so plain silly that I won't answer any more to your posts.

[F.Ultra]

Ad
a) That ton of stuff does not exist. All disk manufacturers that are left today know how to make a good magnetic coating.

b) Are you kidding? No manufacturer would survive if they would not take care of that.

c) Regarding bitrot does not depend on the platter resting or rotating. It's temperature might have more influence. The danger of this effect really comes from the situation that a huge part of the data gets written once and rests on the platter for decades.
Writing the whole partition as I described remedies that problem.

a) yes they do, they have e.g different sensors for vibration and what not that does not exist on normal consumer grade hdds, one of the reason why the MTBF is so much higher on these than on the consumer versions.

b) you do realise that this number indicates that they are fully aware that there is a chance that the same read does not return the exact same data every time? Now that is a very low chance (since again this is a server grade hdd) but it is still there and it is a number they have figured out what it is (since their other drivers have different numbers).

c) data stored on hdds are via magnetic fields so unplugging the drive for 10-20 years have a high chance of altering that magnetic field. This is very simple physics. Not sure why you think constantly having to rewriting the whole partition is somehow a better solution than simply adding checksums ala zfs, btrfs and bachefs, but hey you do you.

I have been driving a car for 31 years now and have never ever had a need for either a seatbelt nor an air-bag, going by your logic I should now rant on car forums that car crashes is a myth...

[LinAdmin]

a) WD Red Pro drives (I have two 8TB of them in my deskop as well as /home) are server grade drives and they have tons of stuff added to avoid bit rot ....

b) The very fact that they even state such a number shows that WD knows about bitrot.

c) However as a test you should power them off and put them in a cupboard for 10-20 years and then power them up and redo the scans again to see what have happened.

Ad
a) That ton of stuff does not exist. All disk manufacturers that are left today know how to make a good magnetic coating.

b) Are you kidding? No manufacturer would survive if they would not take care of that.

c) Regarding bitrot does not depend on the platter resting or rotating. It's temperature might have more influence. The danger of this effect really comes from the situation that a huge part of the data gets written once and rests on the platter for decades.
Writing the whole partition as I described remedies that problem.

[F.Ultra]

I have a bunch of WD-red 8TB disks running in my Server since 2018 (5 years ago) using BtrFS which when scrubbing would never show bitrot. (There has never been any such sign and all data has been perfect).

Since many files are rarely written, I have now unmounted the BtrFS partitions and then refreshed all sectors of each partition using my ecp binary (comparable to cp):

ecp -v /dev/sdb3/ /dev/sdb3/

This took 30 hours per partition and after mounting I ran another scrubbing without errors.

WD Red Pro drives (I have two 8TB of them in my deskop as well as /home) are server grade drives and they have tons of stuff added to avoid bit rot and their Non-recoverable errors per bits read is <1 in 10^15. The very fact that they even state such a number shows that WD knows about bitrot.

However as a test you should power them off and put them in a cupboard for 10-20 years and then power them up and redo the scans again to see what have happened.

That's not an incident report about data loss due to bitrot with ext4, that's just an article on the advantages of COW, specifically cheerleading the use of btrfs. I probably used btrfs before nearly anyone else here, as I was a dedicated SuSE Professional user from the early 2000's through about 2018 while the rest of you goobers were using Ubuntu and Windows and Hannah Montana Linux and so forth.

bitrot, if it's real (unlikely) and not just data loss due to some scriptkiddie setting up raid based on google searches (likely) or using crap cables (highly likely), is just one more of the many reasons to have an ironclad backup plan. It's no reason to avoid ext4, which is one of the most performant desktop file systems available to us.

Backup is not a solution to bitrot and just as cynic wrote once you encounter bitrot you will most likely also discover that all of your backups have them as well. And if we talk about backups I have personally experienced tons of bitrot on backup tapes.

[F.Ultra]

it's not only bitrot. I once had a sata cable going bad after a year. I only noticed it through zfs checksum errors.

Bad sata cables and checksum errors are so common that this is always the first advice someone get when asking for checksum errors in zfs reddit

that too yes, and bad sata/raid cards, if I remember they had a faulty raid card at CERN that they discovered using zfs. Good luck running something other than zfs, btrfs or bcachefs in any of those conditions

[LinAdmin]

F.Ultra[/B] View Post]
Zero of those articles can be serious. Bitrot is a physical phenomenon that is no mystery at all, bits on the drive are not carved in stone (and even things carved in stone experiences bitrot eventually). [...] if people are now claiming that there have never been unrecoverable files on storage media then I have more than one bridge to sell to those.

I have a bunch of WD-red 8TB disks running in my Server since 2018 (5 years ago) using BtrFS which when scrubbing would never show bitrot. (There has never been any such sign and all data has been perfect).

Since many files are rarely written, I have now unmounted the BtrFS partitions and then refreshed all sectors of each partition using my ecp binary (comparable to cp):

ecp -v /dev/sdb3/ /dev/sdb3/

This took 30 hours per partition and after mounting I ran another scrubbing without errors.

[cynic]

That's not an incident report about data loss due to bitrot with ext4, that's just an article on the advantages of COW,

oh good morning! I wrote you two times that the bitrot is not an issue of ext4 or any other fs.
I'm happy that finally you got it.

I probably used btrfs before nearly anyone else here, as I was a dedicated SuSE Professional user from the early 2000's through about 2018 while the rest of you goobers were using Ubuntu and Windows and Hannah Montana Linux and so forth.

no, noob, you didn't used it before me.
I used to compile it by myself into the kernel long before it was officially merged.

bitrot, if it's real (unlikely) and not just data loss due to some scriptkiddie setting up raid based on google searches (likely) or using crap cables (highly likely), is just one more of the many reasons to have an ironclad backup plan. It's no reason to avoid ext4, which is one of the most performant desktop file systems available to us.

look, my English is crappy, I know it.

but at this point, if you still haven't got the point of the whole discussion I think the fault is on you, not me.

[LinAdmin]

Why no ZFS also included in the tests?

Especially knowing that the tests were made on Ubuntu where ZFS is available ...

Yes, I hope that in the next tests it will be included.
BtrFS, ZFS and BcacheFS have the very important crc-checks of all data written.

[andyprough]

don't know what search engine you're using, but one of the first results I get on the topic is this interesting article from 2014: https://arstechnica.com/information-...n-filesystems/

That's not an incident report about data loss due to bitrot with ext4, that's just an article on the advantages of COW, specifically cheerleading the use of btrfs. I probably used btrfs before nearly anyone else here, as I was a dedicated SuSE Professional user from the early 2000's through about 2018 while the rest of you goobers were using Ubuntu and Windows and Hannah Montana Linux and so forth.

bitrot, if it's real (unlikely) and not just data loss due to some scriptkiddie setting up raid based on google searches (likely) or using crap cables (highly likely), is just one more of the many reasons to have an ironclad backup plan. It's no reason to avoid ext4, which is one of the most performant desktop file systems available to us.