Announcement

Collapse
No announcement yet.

Tuxera Claims NTFS Is The Fastest File-System For Linux

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • @kebb

    Btw. it seems you know a lot about ZFS. What about the data which is stored on the drive, but is accessed after very long time (like few months)? ZFS checks its integrity when data is read back, right? There are chances the good data were lost during that time. How ZFS will handle this? Thanks to Oracle's support there's possibility to save such data on Linux. I think they meant such protection when they were saying this:

    The 2.6.27 Linux kernel got bolstered today by "block I/O data integrity infrastructure" code which is seen by Oracle, the code's contributor, as a first for any operating system.
    What about ZFS? It seems it cannot save you from such scenario thus your thesis is obsolete.

    Comment


    • Originally posted by crazycheese View Post

      *1* Platter surface demagnetization errors! SMART detect this.
      Isn't data on the drive checksummed for each sector? Single bit errors and bit rot should be reported as read errors by the hardware.
      Last edited by misiu_mp; 29 June 2011, 05:55 AM.

      Comment


      • Originally posted by misiu_mp View Post
        Isn't data on the drive checksummed for each sector? Single bit errors and bit rot should be reported as read errors by the hardware.
        No, you should read my posts. If you were correct, then CERN would never report any Silent Corruption issues. But there are lots of Silent Corruption in several research reports.

        To summarize: modern disks devote 25% of it's surface to error correcting codes. During usage, lots of errors occur. Most of the errors are corrected in flight, but some errors are difficult and complex, those errors can not be corrected. Every 10^16 bits are not correctable (read the spec sheet of a new SAS Server Enterprise disk that is designed for reliability and safety of data, even SAS disks can not correct all errors). Some of those errors are not even detectable, neither the hardware nor the OS detects the errors.

        I have taken all the information above, from this link. Here you can read much more on this problem, and there are also several research papers:
        Last edited by kebabbert; 30 June 2011, 07:28 AM.

        Comment


        • Originally posted by kraftman View Post
          @kebb

          Btw. it seems you know a lot about ZFS. What about the data which is stored on the drive, but is accessed after very long time (like few months)? ZFS checks its integrity when data is read back, right? There are chances the good data were lost during that time. How ZFS will handle this?
          ZFS will check integrity when reading data, yes. If you dont read data, ZFS will not check integrity.

          But, you can force read of data with "scrub". It is similar to "fsck", but you can scrub in the background while you use the zfs raid. "fsck" requires you to unmount the filesystem and you can not use the filesystem while you check your data with fsck. But, fsck only checks metadata, it does not check the actual data which might be corrupted. For instance, someone said he did fsck on XFS on a 4TB raid, which took like 1 minute or so. This means he read something like 1GB/sec or another huge number. That is not possible to do. This means XFS fsck should not be trusted, because it did not read every sector. ZFS reads every sector, and "scrub" can take some time to do. But you can use your data meanwhile. There is no "fsck" in ZFS, there is only scrub.

          You should do scrub every month if you have server Enterprise disks. And every week, if you have ordinary SATA disks. More often with commodity disks, because they have lower quality than server disks. Preferably with crontab.

          Because ZFS has checksums everywhere, ZFS immediately detects errors. For instance, read this short story what ZFS can do. I doubt any Linux solution can do this?


          ZFS automatically repairs the detected errors. ZFS even detects Silent Corruption (which Linux filesystems can not detect, how can Linux repair Silent Corruption when Linux can not detect the errors?). ZFS repairs the errors if some kind of redundancy is provided. This means ZFS needs to have raid setup, so every sector is copied to another disk. If one sector is corrupt, ZFS read from another disk. If you have no redundancy (only a single disk), ZFS can not repair the error, but ZFS will always detect the error and inform you of the error.

          However, if you have a single disk, you can specify "copies=2" which means every sector will be doubled on a single disk. This halves the storage capacity of the disk, because every data exists in two different sectors. But this gives safety if you have one disk. I dont think any Linux filesystems has this feature on a single disk. Linux needs two disks or more.




          Originally posted by kraftman View Post
          Thanks to Oracle's support there's possibility to save such data on Linux. I think they meant such protection when they were saying this:

          "...The 2.6.27 Linux kernel got bolstered today by "block I/O data integrity infrastructure" code which is seen by Oracle, the code's contributor, as a first for any operating system..."
          It is a great feature. Or is it? As I have said, all filesystems and hardware raid tries to give data safety, but no one succeeds according to research. Hans Reiser tried heavily to give data safety, but failed. So, this "block I/O data integrity infrastructure" - is it safe? Have the engineers succeeded? Are there research on this? Lack of research does not prove it to be safe.

          If I have an antivirus program, and it never trigger warnings - does it mean I never get virus, or does it mean the antivirus program is bad? Lack of warnings does not mean I am completely safe. Maybe there are viruses, that my computer does not notice? Lack of research does not prove something is safe.




          Originally posted by kraftman View Post
          What about ZFS? It seems it cannot save you from such scenario thus your thesis is obsolete.
          It has not been proven that the Linux solution works well. I understand the Oracle engineers tried hard, but did they succeed?

          And even if there is research on Oracle Linux, it does not prove that ZFS is bad. ZFS research shows ZFS to be safer than any other filesystem today.

          If there was Oracle Linux research which shows Oracle Linux is safe, it does not make ZFS unsafe. The ZFS research is still valid. In that case, ZFS and Oracle Linux would be safe according to research. But today, research only shows ZFS to be safe. Maybe Oracle Linux is safe, we dont know yet.

          Comment


          • Originally posted by kraftman View Post
            Wrong. While benchmarks were showing some numbers your conclusions were stupid and irrelevant. I didn't say a sap benchmark is a lie or FUD. You're just manipulating.
            You are saying that SAP benches should be ignored because it is SUN propaganda and not to be trusted. Here is one post where you say that
            Discussion of *BSD operating systems and software, including but not limited to FreeBSD, DragonflyBSD, OpenBSD, and NetBSD. Mac OS X, GNU Hurd, and other alternative operating systems can also be discussed.




            I've showed you link where Linux devs says Bonwick (from SUN) does FUD. That's enough.
            Bonwick (the creator of ZFS) did not FUD. He said that Linux scales bad, and he is correct. He talked about SMP vs HPC. And there are no big Linux SMP servers. The biggest Linux SMP server is 8 cpus, as I know. There are no bigger, as I know. Why is that? Because Linux is bad on SMP servers. Show me a SMP server with more than 8 cpus, if you can. Go ahead.

            Basically SMP is one big server with as many as 64 cpus. IBM AIX and HP-UX and Solaris servers are SMP servers with 64 cpus. IBM biggest Unix box, has only 32 POWER7 cpus because IBM AIX does not scale well enough in SMP servers.

            HPC is a large cluster (a network with lots of PCs) and Linux scales very good on a cluster. There are large clusters, for instance SGI Altix server with 1024 cores. But it is easy to scale on a cluster.

            When people say Linux scales bad, they are refering to SMP servers that are used in the Enterprise. There are no big SMP servers running Linux successfully. There are only Linux clusters in the market today.

            Even Linux scaling experts from RedHat, etc said that Linux scales very well in a cluster (for instance, Google has 10.000 nodes in a network) but they also admitted that Linux scales bad on SMP servers. They said "the true strength of Linux is horizontal scaling" and they talked about Google's big cluster. You want to see the article by the Linux scaling experts from RedHat, and other Linux companies?

            Why are there no big SMP Linux servers on the market? Because Linux scales bad on SMP servers. The SMP server market belongs to Enterprise Unix: AIX, Solaris, HP-UX, IBM Mainframes, etc.



            I showed you papers. Even in this thread. You claimed lots of things, most of it were lies.
            I am talking about RESEARCH papers. You showed White papers. I can show you white paper where Microsoft says that Linux is much more expensive than Windows. The reason? Linux is run on an IBM Mainframe costing million dollars, and Windows is run on a PC, but this fact is hidden and I had to dig to find this fact.



            Are you able to point to that? Maybe I meant it scales to 64 CPUs, but then it sucks?
            Nope, you did not say that. You dont remember what you said? There were lot of people explaining that Solaris servers where sold with 106 cpus, and 144 cpus and still you rejected all of us and you stubbornly said:

            "...Solaris can scale only up to 64 physical CPUs on a single machine while Linux can scale up to 4096 physical CPUs. I don't care about hyper threading etc. I also don't care I could buy a [Solaris] 106 CPUs server years ago and I don't care how many threads Solaris sees..."



            Nope, but saying Linux doesn't scale makes you a lier and FUDer. You've never gave any papers showing that Linux doesn't scale.
            There is no research on this, as I know of. But I have many times explained the difference between scaling on a single huge server (HPC) and scaling on a cluster. Linux scales very well in a cluster, everyone agrees on this. But Linux scales bad on a SMP server, even RedHat scaling experts agree on this.

            If Linux scales well on a SMP server, as you claim: show me a big Linux SMP server that is sold on the market. SMP servers are traditionally for Enterprise use and they cost very much. For instance, IBMs AIX server P595 costed 35 million USD when they got the TPC-C record earlier (until Oracle took the record). If Linux scales good on a single huge server, then everyone should buy cheap Linux SMP servers instead of forcing to buy expensive gear from IBM, HP, Oracle. So, where is the Linux SMP servers with 32 cpus? Show me. There are none. Why?



            The lies.
            I have showed you white papers earlier, and you rejected my white papers. I also showed you research papers, PhD thesis, etc. You have never showed me any research papers. If you did, link to that post here, and I will apologize. Go ahead, show me the post where you link to research papers. If you can not post such a link, then I am talking truth, and it is you that is the liar.



            Haha, but I wanted papers which backup your claims. You were FUDing and lying a lot thus I was asking for papers.
            I did not FUD, nor lie. I posted lot of research papers, but you denied them to exist. I can link to posts where I post research papers, and where you just ignored them. Do you want me to link to those posts? You say something like "no, there are no research papers, where are them, show me the papers". And I have posted those research papers repeatedly, but still you deny them.


            You're still lying, because I don't pretend I don't see the paper. While I was talking about sdc this sounds strange.
            You pretended to not see all the research papers I posted to. I am not lying, you just ignored my posts and asked me to post research papers. Do you want me to quote you on this?



            This time you made some little research.
            What research? Every research paper I posted here, are the same I posted in the old thread. There are no new research papers in this thread. It is the same research papers I posted again and again. I have not looked for new research papers, it is the same old papers I always posted (that you ignored).

            Comment


            • Originally posted by kebabbert View Post
              It is a great feature. Or is it? As I have said, all filesystems and hardware raid tries to give data safety, but no one succeeds according to research. Hans Reiser tried heavily to give data safety, but failed. So, this "block I/O data integrity infrastructure" - is it safe? Have the engineers succeeded? Are there research on this? Lack of research does not prove it to be safe.
              Yeah, but this research was done, before someone submited patches. I believe they succeeded.

              If I have an antivirus program, and it never trigger warnings - does it mean I never get virus, or does it mean the antivirus program is bad? Lack of warnings does not mean I am completely safe. Maybe there are viruses, that my computer does not notice? Lack of research does not prove something is safe.
              Like I said many times it's matter of believing. I have to believe the research is still right about ZFS etc.

              Comment


              • Originally posted by kebabbert View Post
                You are saying that SAP benches should be ignored because it is SUN propaganda and not to be trusted. Here is one post where you say that
                http://phoronix.com/forums/showthrea...096#post104096
                I was talking about sun's blog. It's very sad you have problems with obvious things.

                Bonwick (the creator of ZFS) did not FUD. He said that Linux scales bad, and he is correct. He talked about SMP vs HPC. And there are no big Linux SMP servers. The biggest Linux SMP server is 8 cpus, as I know. There are no bigger, as I know. Why is that? Because Linux is bad on SMP servers. Show me a SMP server with more than 8 cpus, if you can. Go ahead.
                He did FUD according to Linux dev response. I also consider he did FUD. He wasn't correct saying Linux scales bad. Linux dev said Linux scales crap out of Solaris, so I believe him. You're still lying: "The biggest Linux SMP server is 8 cpus, as I know. There are no bigger, as I know." You gave a SAP paper where there's a 48 core SMP Linux system, so you knew about this, so you're lying.

                Basically SMP is one big server with as many as 64 cpus. IBM AIX and HP-UX and Solaris servers are SMP servers with 64 cpus. IBM biggest Unix box, has only 32 POWER7 cpus because IBM AIX does not scale well enough in SMP servers.
                Like I said before.

                HPC is a large cluster (a network with lots of PCs) and Linux scales very good on a cluster. There are large clusters, for instance SGI Altix server with 1024 cores. But it is easy to scale on a cluster.
                There's even bigger - 2048 core machine and the funny thing a single Linux kernel is running on it.

                When people say Linux scales bad, they are refering to SMP servers that are used in the Enterprise. There are no big SMP servers running Linux successfully. There are only Linux clusters in the market today.
                You're data is very old. You're still talking about Linux 2.4.

                Even Linux scaling experts from RedHat, etc said that Linux scales very well in a cluster (for instance, Google has 10.000 nodes in a network) but they also admitted that Linux scales bad on SMP servers. They said "the true strength of Linux is horizontal scaling" and they talked about Google's big cluster. You want to see the article by the Linux scaling experts from RedHat, and other Linux companies?
                The same as above.

                Why are there no big SMP Linux servers on the market? Because Linux scales bad on SMP servers. The SMP server market belongs to Enterprise Unix: AIX, Solaris, HP-UX, IBM Mainframes, etc.
                Oh, there are. Like those in SAP benchmarks. Btw. do you have any info about market share?

                I am talking about RESEARCH papers. You showed White papers. I can show you white paper where Microsoft says that Linux is much more expensive than Windows. The reason? Linux is run on an IBM Mainframe costing million dollars, and Windows is run on a PC, but this fact is hidden and I had to dig to find this fact.
                So, you were lying saying I didn't show you the papers? If you're aware such comparision is meaningles then why are you doing the same regarding SMP scaling?

                Nope, you did not say that. You dont remember what you said? There were lot of people explaining that Solaris servers where sold with 106 cpus, and 144 cpus and still you rejected all of us and you stubbornly said:

                "...Solaris can scale only up to 64 physical CPUs on a single machine while Linux can scale up to 4096 physical CPUs. I don't care about hyper threading etc. I also don't care I could buy a [Solaris] 106 CPUs server years ago and I don't care how many threads Solaris sees..."
                I corrected myself later saying I was according to h-online.



                There is no research on this, as I know of. But I have many times explained the difference between scaling on a single huge server (HPC) and scaling on a cluster. Linux scales very well in a cluster, everyone agrees on this. But Linux scales bad on a SMP server, even RedHat scaling experts agree on this.
                Still, an old data.

                If Linux scales well on a SMP server, as you claim: show me a big Linux SMP server that is sold on the market. SMP servers are traditionally for Enterprise use and they cost very much. For instance, IBMs AIX server P595 costed 35 million USD when they got the TPC-C record earlier (until Oracle took the record). If Linux scales good on a single huge server, then everyone should buy cheap Linux SMP servers instead of forcing to buy expensive gear from IBM, HP, Oracle. So, where is the Linux SMP servers with 32 cpus? Show me. There are none. Why?
                Like before, there's even one in your SAP papers 48 cores SMP Linux machine.

                I have showed you white papers earlier, and you rejected my white papers.
                I probably rejected your idiotic conclusions.

                I also showed you research papers, PhD thesis, etc. You have never showed me any research papers. If you did, link to that post here, and I will apologize. Go ahead, show me the post where you link to research papers. If you can not post such a link, then I am talking truth, and it is you that is the liar.
                Show me I was talking about research papers and not about white papers.

                I did not FUD, nor lie. I posted lot of research papers, but you denied them to exist. I can link to posts where I post research papers, and where you just ignored them. Do you want me to link to those posts? You say something like "no, there are no research papers, where are them, show me the papers". And I have posted those research papers repeatedly, but still you deny them.
                I wanted papers which backup your claims like I said many times. However, you lie a lot.

                You pretended to not see all the research papers I posted to. I am not lying, you just ignored my posts and asked me to post research papers. Do you want me to quote you on this?
                Go ahead and show me the papers which proves your claims.

                What research? Every research paper I posted here, are the same I posted in the old thread. There are no new research papers in this thread. It is the same research papers I posted again and again. I have not looked for new research papers, it is the same old papers I always posted (that you ignored).
                Everything's good?

                Comment


                • Originally posted by kraftman View Post
                  You're still lying: "The biggest Linux SMP server is 8 cpus, as I know. There are no bigger, as I know." You gave a SAP paper where there's a 48 core SMP Linux system, so you knew about this, so you're lying.
                  I'm sorry, my fault. It has only 8CPUs, so here's a correct example:



                  You knew about this too, so what I wrote is correct.

                  Comment


                  • I'm confused X.x 48 cores is trivial these days 12 cores per die is off the shelf stuff... and quad socket mobos aren't that uncommon. I even know a fellow that has at 32 core box with quad channel ram on a relativly light budget as well its quite a beast for a serverish/desktop.

                    And anyone wanting data integrity is going to be running raid as there isn't anything saying that a whole drive couldn't go kaput. noone will rely on the FS alone to save them...

                    Comment


                    • Originally posted by cb88 View Post
                      And anyone wanting data integrity is going to be running raid as there isn't anything saying that a whole drive couldn't go kaput. noone will rely on the FS alone to save them...
                      The point isn't really for the FS alone to save you from a hardware failure, it's that there are failure modes in which you won't even know that a failure happened unless something is hashing the data on a regular basis. It doesn't have to be the filesystem, but that's arguably a reasonable place to do it (and ZFS is really more like a combination of filesystem/volume management/softraid than just a FS anyway).

                      Comment

                      Working...
                      X