Originally posted by kebabbert
View Post
Announcement
Collapse
No announcement yet.
Oracle Plans To Bring DTrace To Linux
Collapse
X
-
-
Originally posted by Zetbo View Post
Larry Ellison said that Linux is for lowend, and Solaris is for Highend. I cant find that link now, but if you want, I can google for that link. He also said
Ellison now calls Solaris "the leading OS on the planet,"
But I am surprised that DTrace comes to Linux, because DTrace is much better than anything Linux has. Does really Larry want Linux to be as good as Solaris?
There is a huge technical post by a famous DTrace contributor, where he compares Systemtap to DTrace. Among others, he says that Systemtap might crash the server which makes it not usable for production servers. Whereas DTrace is safe and can be used in production servers. He goes on and on, and it seems that Systemtap has missed some of the main points of having a dynamically instrument like DTrace.
But if DTrace comes to Linux, that is good for Linux. Here is the main architect of DTrace, trying out Linux DTrace.
Comment
-
Originally posted by simcop2387 View PostZFS will always be able to detect data corruption (I believe), but repair will only happen if you are using one of the many RAID-Z configurations. Admittedly though if you're using ZFS and NOT running with RAID-Z then you really shouldn't be managing servers.
Comparing ZFS to EXT4 though is rather unfair. EXT4 was as you say never designed for that protection. A more correct comparison would be to compare UFS to EXT4, as they both serve the same purpose. And before you say that you shouldn't ever use UFS, there are times where the overhead of ZFS shouldn't be bothered with, say when you are running many virtual solaris systems with all of their storage already on ZFS. There's no reason to put ZFS inside of ZFS.
Comment
-
Originally posted by kraftman View PostThis is meaningless. ZFS is not safe by design, because in scenarios when there are long pauses between fs checks it can be corrupted. I showed you this once.
Regarding if there are long pauses between fs checks, no, that does not matter. If you dont do a ZFS fs check in one year (as I have), what happens is that ZFS always detects corruption. Detection is an ongoing process, that never stops. If ZFS detects corruption, ZFS will automatically repair the corruption (if you have redundancy like raid or "copies=2") when you request the data.
"I've been running over a week now with a faulty setup which is still corrupting data on its way to the disk, and have yet to see a problem with my data, since ZFS handily detects and corrects these errors on the fly."
This guy has a weak power supply, and ZFS detects corruption in his setup almost immediately. Earlier, the earlier filesystem he used, did not detect these problems. He ran for years without no one telling him that data corruption occured all the time. But ZFS notices the slightest data corruption, and tells the user, and automatically repairs corruption.
But every once in a while, you should traverse everything on disk and check everything, yes. Do a fs check.
Originally posted by kraftman View PostYou can only say ZFS is safer by default. There are ways than can make Ext4 nearly completely safe. Those 'computer science researches' didn't prove this, but they showed it has some mechanisms that can help in fighting silent data corruption.
The computer science researchers did prove that for every artifical error they injected into the disks, ZFS detected all errors. And also, ZFS repaired all errors, when there were redundancy (for instance, raid). Other computer science researchers also injected artifical errors on common Linux filesystem disks, and the Linux filesystems did not even detect those errors. How can Linux filesystems repair errors they can not even detect? First step is detection.
To detect errors, ZFS knows about everything; from RAM down to disk. The whole chain. ZFS has control over everything, ZFS is raid, filesystem, etc in one piece of software. This gives ZFS the ability to do end-to-end checksums: "The data in RAM, is it identical to what was stored on disk?" This comparison requires ZFS to have control over everything.
Have you played a game as kid? You whisper a word in one child, he whispers it to the other, etc. The last child says the word loud and everyone giggles, because the starting word and the last word are never the same. You need always to compare the start, to the end - to be sure the data is identical. ZFS does this, compares RAM to disk. End-to-end checksums. This is because ZFS is one piece of code, doing everything.
And guess what? Linux has several different layers, one raid layer. One filesystem layer. etc. All these layers does not know about other layers. There are no one to do end-to-end checksums. No one can compare RAM to disk, because there are several independent Linux layers.
Maybe you heard about Linux kernel developers mocking ZFS for "rampant violation of layers"? The reason ZFS can do end-to-end checksums, and Linux can not - is because ZFS violates layers! That is the main point of ZFS. And for this, Linux kernel developers did not understand anything of ZFS and said ZFS was a piece of shit, because it violates layers. Well, because ZFS violates layers - and Linux does not - ZFS is superior. Again Linux kernel developers show their ignorance. Because Linux kernel developers complain on the Linux code is "having low quality" - maybe there is a correlation between ignorance and low code quality?
ZFS does great things because it violates layers. The ZFS dev team realized the need to violate layers, and discusses this issue in several interviews and documents. Any attempt to clone ZFS will need to violate layers, too. Just like BTRFS - which also violates layers.
Question 1) How did you know what computer science researchers say about ZFS? Have you studied the latest research or what? Which research papers have you read?
Question 2) You have many times said "It is widely known Oracle wants to kill old, crappy, legacy slowlaris". Can you provide links, or are you just trying to start evil rumours and doing FUD about Solaris? (Which you have confessed earlier).
I have never seen that Larry Ellison wants to kill Solaris. He says it is the best Unix out there.
On the other hand, IBM has offically said they are going to kill of AIX, the Unix version from IBM. This is not evil rumours nor FUD from me. Here is the link going to IBM executives. Thus, I am not spreading FUD, I speak true and always back up my claims. I have substance in my claims. Check my sources, yourself:
ZDNET news and advice keep professionals prepared to embrace innovation and ready to build a better future.
"The day is approaching when Linux is likely to replace IBM's version of Unix, the company's top software executive said, an indication that the upstart operating system's stature is rising within Big Blue....Asked whether IBM's eventual goal is to replace AIX with Linux, Mills responded, "It's fairly obvious we're fine with that idea...It's the logical successor."
Well, I don't care about PR bull and crappy and biased benchmarks against very old Linux systems. System that has 30% slower binaries with highest optimization simply can't be fast, can it? The reality shows slowlaris isn't interesting for Oracle, but they keep it because of old sun (thankfully's dead) customers. As for btrfs being 64bit system that's true and that's very good design choice.
And regarding performance of ZFS vs BTRFS. Well, ZFS scales better than BTRFS, and ZFS is faster when you start to use many disks. Here is a benchmark of only 16 SSD disks on BTRFS vs ZFS. And we see that out-of-the-box, ZFS is faster than BTRFS. Sure, BTRFS might be faster on a single disk, but Solaris has always targeted big servers, big scalability - and never really cared for a single disk or quad cores. Solaris is for many disks, many cpus, etc. The more disks you have, the better ZFS will be, and the slower BTRFS will be. Same with cpus.
Regarding BTRFS being 64 bit, that is quite bad actually. I dont have the time to redo the calculations again now, but this is reasoning from ZFS developers. They reasoned something like this:
Today, we are storing Petabytes of data. CERN are storing petabytes, Facebook, Google, etc. To store PB of data, requires something like 2^62 bits. In a couple of years, all data will be doubled, thus requiring 2^63 bits. And in another few years, it will require 2^64 bits. After that, 64 bit filesystems will not do. Then you need filesystems that can store more data than 64 bits. Maybe ZFS would be a 72bit filesystem? And in a few years, ZFS would need to be increased again. Instead, ZFS developers said, "well, let us make 128 bits, and then we never have to worry anymore". Thus, ZFS is 128 bit and can handle 2^128 bits. 128 bit is enough and will never need to be increased. To fill up 2^128 bit, you need something like more atoms in the whole earth or something similar. If you stapled the entire earth with 4TB disks, 10 meters high everywhere on land and on sea - that would be something like 2^100 bits of storage or something similar. You would need several earths like that, to reach 2^128 bits. Thus, 128 bit filesystems is all that humankind will ever need. Laws of physics say so.
Therefore, in some years from now, they need to redesign BTRFS to be more than 64 bits. That is short sighted, and not future proof. This again shows why BTRFS has some bad design choices and why BTRFS is inferior to ZFS. Heck, even one RedHat developer said BTRFS is "broken by design".
Comment
-
Originally posted by kebabbert View PostYou did? I missed that. Can you please post it again?
Regarding if there are long pauses between fs checks, no, that does not matter. If you dont do a ZFS fs check in one year (as I have), what happens is that ZFS always detects corruption. Detection is an ongoing process, that never stops. If ZFS detects corruption, ZFS will automatically repair the corruption (if you have redundancy like raid or "copies=2") when you request the data.
If you make ext4 "nearly completely safe", then it has to be heavily modified, and will be very similar to ZFS design in data detection aspect. And if there are ways to make ext4 nearly completely safe, why dont they do it?
And guess what? Linux has several different layers, one raid layer. One filesystem layer. etc. All these layers does not know about other layers. There are no one to do end-to-end checksums. No one can compare RAM to disk, because there are several independent Linux layers.
Question 1) How did you know what computer science researchers say about ZFS? Have you studied the latest research or what? Which research papers have you read?
Question 2) You have many times said "It is widely known Oracle wants to kill old, crappy, legacy slowlaris". Can you provide links, or are you just trying to start evil rumours and doing FUD about Solaris? (Which you have confessed earlier).
I have never seen that Larry Ellison wants to kill Solaris. He says it is the best Unix out there.
And regarding performance of ZFS vs BTRFS. Well, ZFS scales better than BTRFS, and ZFS is faster when you start to use many disks. Here is a benchmark of only 16 SSD disks on BTRFS vs ZFS. And we see that out-of-the-box, ZFS is faster than BTRFS. Sure, BTRFS might be faster on a single disk, but Solaris has always targeted big servers, big scalability - and never really cared for a single disk or quad cores. Solaris is for many disks, many cpus, etc. The more disks you have, the better ZFS will be, and the slower BTRFS will be. Same with cpus.
http://www.mail-archive.com/linux-bt.../msg05647.html
Regarding BTRFS being 64 bit, that is quite bad actually. I dont have the time to redo the calculations again now, but this is reasoning from ZFS developers. They reasoned something like this:
Today, we are storing Petabytes of data. CERN are storing petabytes, Facebook, Google, etc. To store PB of data, requires something like 2^62 bits. In a couple of years, all data will be doubled, thus requiring 2^63 bits. And in another few years, it will require 2^64 bits. After that, 64 bit filesystems will not do. Then you need filesystems that can store more data than 64 bits. Maybe ZFS would be a 72bit filesystem? And in a few years, ZFS would need to be increased again. Instead, ZFS developers said, "well, let us make 128 bits, and then we never have to worry anymore". Thus, ZFS is 128 bit and can handle 2^128 bits. 128 bit is enough and will never need to be increased. To fill up 2^128 bit, you need something like more atoms in the whole earth or something similar. If you stapled the entire earth with 4TB disks, 10 meters high everywhere on land and on sea - that would be something like 2^100 bits of storage or something similar. You would need several earths like that, to reach 2^128 bits. Thus, 128 bit filesystems is all that humankind will ever need. Laws of physics say so.
Therefore, in some years from now, they need to redesign BTRFS to be more than 64 bits. That is short sighted, and not future proof. This again shows why BTRFS has some bad design choices and why BTRFS is inferior to ZFS. Heck, even one RedHat developer said BTRFS is "broken by design".
Comment
-
Originally posted by kraftman View PostNo, you didn't miss it and no, I won't post it once again.
How can I know if you are telling the truth now? If you can not post a simple link again, then it looks like FUD. Dont you think?
I know you have showed links earlier, but none of the links where relevant. In one case you showed a link, where they compared an old 800MHz SPARC Solaris server, vs a dual core Intel 2.4GHz Linux, and your link claimed that Linux is faster than Solaris. You thought that link was good and relevant. I say your link was not relevant. If you install Linux and Solaris on the same hardware - that is interesting and relevant - when we discuss performance of Solaris vs Linux. Thus, your links are not good. This link you showed earlier, can you repost it, or is the link as meaningless as your earlier links?
That's the problem. You've got to have copies to repair file system.
1) ext4 has no "copies=2" command, ext4 needs another disk to be able fetch missing data. ext4 must have two disks, or more. ZFS can have a single disk.
2) ext4 has no way of detecting corrupted data or repair it. ext4 does parity calcuations, but no checksum for detecting corrupted data.
So I dont see it as a problem that ZFS can repair corrupted data, whereas ext4 can not. What is the problem that ZFS can repair corrupted data? Can you explain again?
The same can be done with Ext4.
There are also bugs in zfs, so your data is not completely safe with it.
Many people believe that ZFS is the safest alternative on the market today. Not a completely safe, but the safest. ext4 has no data corruption detection at all, so it is not safe at all. I fail to see how "ext4 is near completely safe" - can you describe how, or are FUDing again?
Those who want to make it such safe just use proper system configuration and copies (raid). It's not that same file system has to care about data safeness. The same about detection and recovery.
That's no true and I showed you this one, too. Patches were even sent by Oracle btw.
The one you gave.
You spread FUD about Linux,
but I say true things about slowlaris. Links are meaningless in this case which is obvious, but marketshare, popularity and Oracle actions clearly shows slowlaris is going to end.
Larry has said officially that he is increasing resources much more than Sun ever had. There will be more developers on Solaris and SPARC cpu, than Sun ever had.
In other words, Larry says he will bet heavily on Solaris. You say he is going to kill Solaris. This post shows that you are not correct on this, Solaris will not be killed. Do you agree that Larry is not interested in killing Solaris? Why would Larry say that “…Solaris is overwhelmingly the best open systems operating system on the planet.” if he wants to kill Solaris? No, you are not correct on this. I have showed you numerous links where Larry praises Solaris, and still you say that Larry are going to kill Solaris. Why? Isnt what you do, pure FUD and Trolling?
He says many stupid things. How many unixes are out there, today?
Such old benchmarks with unstable btrfs version doesn't matter at all.
64bit is simply enough and will be enough for long time. CERN, Google, Facebook runs Linux not slowlaris, so 64bit is and will be (nothing suggests they plan to replace Linux by system that has 30% slower binaries) enough.
"Having conducted testing and analysis of ZFS, it is felt that the combination of ZFS and Solaris solves the critical data integrity issues that have been seen with other approaches. They feel the problem has been solved completely with the use of this technology. There is currently about one Petabyte of Thumper storage deployed across Tier1 and Tier2 sites. That number is expected to rise to approximately four Petabytes by the end of this summer."
No, that's simply not true and this sounds like sun's FUD. Red Hat employer was mistaken and if you read discussion you should be aware of this.
If CERN uses 64 bits, then CERN needs to split the data so that the data does not go beyond 2^64 bits. So CERN will have some data pools, with 2^64 bits, and another data pool with 2^64 bits, etc. Thus, there will be several data pools, and it will be difficult to examine all data in different pools. It is then better to use one single data pool, that holds all data because then CERN can run all calculations, without having to split them up. Thus, it is true that BTRFS will not be able to handle big scenarios - which means that BTRFS needs to be redesigned to use more than 64 bits. So, yes, I speak true.
Regarding the RedHat developer, he said that BTRFS has some issues. That is true, and I do not lie nor FUD about this. Do you want to see the post where he writes this?Last edited by kebabbert; 06 November 2011, 07:05 PM.
Comment
-
Kebab: ZFS cannot deal with in-RAM corruption - by design. Deal with it.
And you use Larry Ellison's quote "Solaris is overwhelmingly the best open systems operating system on the planet" for evaluating Solaris' technical status. Am sure you don't have a day job.. and don't wanna know details of your night job.Last edited by bluetomato; 10 November 2011, 12:28 PM.
Comment
-
Originally posted by bluetomato View PostKebab: ZFS cannot deal with in-RAM corruption - by design. Deal with it.
I hope you dont believe that any Linux filesystem such as BTRFS corrects errors in the GPU, or in RAM? Are there ANY Linux filesystem that correctly saves data to disk? Research shows that most common Linux filesystems are not safe. There are no research on BTRFS yet, but Kraftman says it is in beta phase so BTRFS should be ruled out. (I showed benches of ZFS vs BTRFS on SSD disks, and Kraftman ruled that bench out because BTRFS is not done yet nor stable)
And you use Larry Ellison's quote "Solaris is overwhelmingly the best open systems operating system on the planet" for evaluating Solaris' technical status.
MY point is that I am quoting Larry Ellison on this, because "some people" says that Larry wants to kill off Solaris, because it is slow (slowlaris). Well, it seems that Larry thinks that Solaris is the best. And I have shown benchmarks where Solaris holds several world records today. So, Larry does bet on Solaris, and Solaris is the fastest in the world on some benchmarks. So I have disproved "some people", yes?
Am sure you don't have a day job.. and don't wanna know details of your night job.Last edited by kebabbert; 10 November 2011, 03:22 PM.
Comment
-
Re Disk capacities...
SGI says this about XFS http://oss.sgi.com/projects/xfs/):
XFS is a full 64-bit filesystem, and thus is capable of handling filesystems as large as a million terabytes.
2^63 = 9 x 10^18 = 9 exabytes
That's WELL under the limit of 64bit FS capacity (1/80), per SGI's numbers.
XFS is one of the things CERN has gone to great lengths to support/use--it ships with SL by default.
Hence the reference.
(Plus it's supposedly the fastest FS for huge data files.)
Comment
-
Originally posted by Ibidem View PostSGI says this about XFS http://oss.sgi.com/projects/xfs/):
CERN expects to need, per kebabbert's link, 57 petabytes of disk and 43 petabytes of tape storage (100 petabytes total).
That's WELL under the limit of 64bit FS capacity (1/80), per SGI's numbers.
XFS is one of the things CERN has gone to great lengths to support/use--it ships with SL by default.
Hence the reference.
(Plus it's supposedly the fastest FS for huge data files.)
Also, I have read that some vendor (is it RedHat) only supports up to 16TB raid sets with XFS. Is this true or false?
Comment
Comment