Announcement

Collapse
No announcement yet.

EXT4 In Linux 3.5 Gets CRC32 Meta-Data

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • phoronix
    started a topic EXT4 In Linux 3.5 Gets CRC32 Meta-Data

    EXT4 In Linux 3.5 Gets CRC32 Meta-Data

    Phoronix: EXT4 In Linux 3.5 Gets CRC32 Meta-Data

    The pull request for the EXT4 file-system in the Linux 3.5 kernel and there's one prominent new feature...

    http://www.phoronix.com/vr.php?view=MTExMTc

  • unknown2
    replied
    Does it checksum metadata only? Is there checksum support for file's data?

    Also, CRC32 seems a little bit weak.
    As far as i know, CRC32 is used by many hardware internally for error correction.
    When data corrupted and undetected in hardware level (which means the hardware CRC32 test probably passed without error), i doubt how reliable that ext4's CRC32 can detect corruption.

    At least an alternative implementation of MD5 should be provided, the overhead should be acceptable if those crypto library are written in SIMD instruction (MMX, SSE)
    Last edited by unknown2; 07-21-2012, 08:35 PM.

    Leave a comment:


  • curaga
    replied
    Via's cpus have supported hardware SHA-1/SHA-256 for years. Many ARM cpus also support it in hw.

    Leave a comment:


  • jabl
    replied
    Modern x86 (and IIRC SPARC) processors have hw accelerated CRC32C. "Luckily", the exact same polynomial is used in the btrfs and ext4 crc checksum, among others, allowing those to benefit from the hw acceleration.

    Leave a comment:


  • uid313
    replied
    Originally posted by smitty3268 View Post
    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.
    These days you have hardware-accelerated cryptography, I know modern Intel processosrs support AES-NI (AES instruction set). Don't know if it do hardware-accelerated hashing though.

    Leave a comment:


  • F.Ultra
    replied
    Originally posted by smitty3268 View Post
    There's a reason every FS and DB I've ever heard of that added checksums uses simple CRC hashing.

    It's a simple algorithm that is well suited to the task.

    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

    If you want to verify that the data hasn't been tampered with, you should just encrypt the whole FS.
    SHA-1 and the various hash based algorithms will give you a much higher error detection rate than a simple crc-32, a crc-32 can only detect certain types of errors and certain degrees of errors within that (i.e it can for example catch burst errors but not if they are over X bits of length). They do introduce a major overhead though, however since this is for the meta-data only anyways, the performance overhead shouldn't be that much.

    Leave a comment:


  • gilboa
    replied
    Hopefully by the time F18/F19 will be released e2fschk will support on-disk upgrade giving an option, via anaconda, to switch existing ext4 to CRC-check-summed.
    As it stands, I doubt that I'll be using btrfs before 2014 and better error detection in existing stable FS is always welcome.

    - Gilboa

    Leave a comment:


  • smitty3268
    replied
    Originally posted by uid313 View Post
    No, SHA-1 is not a cipher, it is a hash algorithm. It does not encrypt anything.

    SHA-1 is very suitable to verify the integrity of data. Not only does it verify that the data is not corrupted, it also verifies the integrity of the data, that it has not been tampered with.
    There's a reason every FS and DB I've ever heard of that added checksums uses simple CRC hashing.

    It's a simple algorithm that is well suited to the task.

    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

    If you want to verify that the data hasn't been tampered with, you should just encrypt the whole FS.

    Leave a comment:


  • uid313
    replied
    Originally posted by oliver View Post
    Don't feed the troll, I'll bite anyhow.

    CRC is used to verify that the data inside the block is not corrupted. it's a redundancy check. Is my super-block valid, or not.

    CRC-64 is not that useful in this specific scenario.

    http://en.wikipedia.org/wiki/Cyclic_redundancy_check

    SHA-1 is completely useless as that is a cipher. The point was not to encrypt the metadata, just to verify it.
    No, SHA-1 is not a cipher, it is a hash algorithm. It does not encrypt anything.

    SHA-1 is very suitable to verify the integrity of data. Not only does it verify that the data is not corrupted, it also verifies the integrity of the data, that it has not been tampered with.
    Last edited by uid313; 06-01-2012, 06:34 PM.

    Leave a comment:


  • oliver
    replied
    Originally posted by uid313 View Post
    What about CRC-64?

    What about using a secure hashing algorithm such as SHA-1 or Skein?
    Don't feed the troll, I'll bite anyhow.

    CRC is used to verify that the data inside the block is not corrupted. it's a redundancy check. Is my super-block valid, or not.

    CRC-64 is not that useful in this specific scenario.

    http://en.wikipedia.org/wiki/Cyclic_redundancy_check

    SHA-1 is completely useless as that is a cipher. The point was not to encrypt the metadata, just to verify it.

    Leave a comment:

Working...
X