Announcement

Collapse
No announcement yet.

EXT4 In Linux 3.5 Gets CRC32 Meta-Data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by smitty3268 View Post
    There's a reason every FS and DB I've ever heard of that added checksums uses simple CRC hashing.

    It's a simple algorithm that is well suited to the task.

    You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.

    If you want to verify that the data hasn't been tampered with, you should just encrypt the whole FS.
    SHA-1 and the various hash based algorithms will give you a much higher error detection rate than a simple crc-32, a crc-32 can only detect certain types of errors and certain degrees of errors within that (i.e it can for example catch burst errors but not if they are over X bits of length). They do introduce a major overhead though, however since this is for the meta-data only anyways, the performance overhead shouldn't be that much.

    Comment


    • #12
      Originally posted by smitty3268 View Post
      You could replace it with SHA-1, but what would that give you? A much more complicated and CPU-taxing algorithm, and nothing else. Crypto hashes have to be a lot more complicated so that small changes to the data provide pseudo-random changes to the hash that can't be reverse-engineered. There's isn't a single good reason to want that overhead in an integrity check.
      These days you have hardware-accelerated cryptography, I know modern Intel processosrs support AES-NI (AES instruction set). Don't know if it do hardware-accelerated hashing though.

      Comment


      • #13
        Modern x86 (and IIRC SPARC) processors have hw accelerated CRC32C. "Luckily", the exact same polynomial is used in the btrfs and ext4 crc checksum, among others, allowing those to benefit from the hw acceleration.

        Comment


        • #14
          Via's cpus have supported hardware SHA-1/SHA-256 for years. Many ARM cpus also support it in hw.

          Comment


          • #15
            Does it checksum metadata only? Is there checksum support for file's data?

            Also, CRC32 seems a little bit weak.
            As far as i know, CRC32 is used by many hardware internally for error correction.
            When data corrupted and undetected in hardware level (which means the hardware CRC32 test probably passed without error), i doubt how reliable that ext4's CRC32 can detect corruption.

            At least an alternative implementation of MD5 should be provided, the overhead should be acceptable if those crypto library are written in SIMD instruction (MMX, SSE)
            Last edited by unknown2; 21 July 2012, 08:35 PM.

            Comment

            Working...
            X