Bcachefs Changes Merged Without Issue For The Linux 6.14 Kernel

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • tytso
    Junior Member
    • Feb 2009
    • 48

    #31
    To be fair, while bcachefs isn't a total solo project -- Kent authored 72% of the patches between 6.11 and 6.12, for example, where as out of the 103 patches to ext4 during the same time period, I authored precisely 0%. That's because I subscribe very firmly to the school of thought which says that programming is a team sport, and my job as tech lead is to enable the ext4 contributors to do their best to improve the file system. We have weekly conference calls, and Darrick Wong, senior XFS developer and former XFS maintainer, attends those calls --- and I've been known to help him out on XFS testing issues[1], and Darrick has helped me out with various ext4 test issues, and has even reviewed an ext4 patch or two. We cooperate with each other, and that's a good thing.

    [1] https://lore.kernel.org/fstests/[email protected]/T/#u

    I'll let other people decide whether they would like to trust their data to someone who is a single hot-shot programmer who might very well be more talented than I on a head-to-head basis --- but I'll give you a hint --- you can "cheat" by bringing a team to bear on a problem. You don't have to do it all by yourself. Of course, in order to do this you need to know how to bring out the best in others, and you have to work together. And being nice to one another on mailing lists doesn't hurt on that score.

    Comment

    • pWe00Iri3e7Z9lHOX2Qx
      Senior Member
      • Jul 2020
      • 1608

      #32
      Originally posted by tytso View Post

      Hardly. There are over a half dozen contributors each kernel release to ext4. These days most my time is spent doing code review and running tests and improving the {kvm,gce,qemu,android}-xfstests test appliance. And I very much rely on 2 or 3 other developers working at SuSE and IBM to help out with the code review.
      I think this overall spirit would be true for xfs and btrfs as well. Even if today's maintainers burned out and walked away (we don't want them to!), these 3 filesystems are so widespread that someone would either volunteer to help, or Red Hat / SUSE would just fund them (even more) directly. The "hit by bus" factor isn't a big deal for ext4 / xfs / btrfs. It's still big for bcachefs, but I hope that changes and we get more contributors helping with bigger changes (scrub, erasure coding, etc.).

      Comment

      • ihatemichael
        Senior Member
        • Jan 2014
        • 301

        #33
        Originally posted by tytso View Post

        Hardly. There are over a half dozen contributors each kernel release to ext4. These days most my time is spent doing code review and running tests and improving the {kvm,gce,qemu,android}-xfstests test appliance. And I very much rely on 2 or 3 other developers working at SuSE and IBM to help out with the code review.
        Are we ever going to get new features on ext4 such as reflinks, etc, or is ext4 set in stone feature-wise?

        One thing that I have noticed when it comes to filesystems is that they don't always go as intended, for example, btrfs was supposed to replace ext4, ext4 is still here, the same probably is going happen with bcachefs and btrfs.

        I don't think that's really a bad thing because I really appreciate ext4 for what it is, a simple filesystem that gets the job done in most cases, I hope it will continue to be developed and improved for a long time.

        Comment

        • Raka555
          Junior Member
          • Nov 2018
          • 682

          #34
          Originally posted by ihatemichael View Post

          Are we ever going to get new features on ext4 such as reflinks, etc, or is ext4 set in stone feature-wise?

          One thing that I have noticed when it comes to filesystems is that they don't always go as intended, for example, btrfs was supposed to replace ext4, ext4 is still here, the same probably is going happen with bcachefs and btrfs.

          I don't think that's really a bad thing because I really appreciate ext4 for what it is, a simple filesystem that gets the job done in most cases, I hope it will continue to be developed and improved for a long time.
          TBH, I would rather like to see them doing an ext5 than changing ext4 into unstable.
          Last edited by Raka555; 21 January 2025, 03:56 PM.

          Comment

          • ihatemichael
            Senior Member
            • Jan 2014
            • 301

            #35
            Originally posted by Raka555 View Post

            TBH, I would rather like to see them doing an ext5 than changing ext4 into unstable.
            That would work too, to me the most enjoyable part about ext4 is the "set and forget" thing about it, there are some things about btrfs that I would like to use, for example: subvolumes and reflinks, but I feel btrfs always comes at a cost, additional complexity to manage and the "cognitive overhead" (having to remember to do this and that to keep the thing working in a good state).

            With ext4 I don't need to think about the filesystem again and that's a good thing in my book, but it would be great to have some of those features without the extra cost of having a more complex filesystem.

            Comment

            • tytso
              Junior Member
              • Feb 2009
              • 48

              #36
              Originally posted by ihatemichael View Post

              Are we ever going to get new features on ext4 such as reflinks, etc, or is ext4 set in stone feature-wise?

              One thing that I have noticed when it comes to filesystems is that they don't always go as intended, for example, btrfs was supposed to replace ext4, ext4 is still here, the same probably is going happen with bcachefs and btrfs.

              I don't think that's really a bad thing because I really appreciate ext4 for what it is, a simple filesystem that gets the job done in most cases, I hope it will continue to be developed and improved for a long time.
              Ext4 does get some new features, but they are ones which companies are willing to fund because the return on investment of developing the feature makes sense from a cost/benefit perspective. So for example, fscrypt and case insensitive directories were features that were useful for Android and Chrome OS, and was funded at least partially by those product groups (Steam also cared about case folding and supported one of the engineers). We're looking to add untorn write support because this would improve database performance for cloud emulated block devices where you can guarantee 16k atomic writes, so you can eliminate double buffering in mysql andpostgres. (In fact this is something Amazon and Google can do with their first-party database products by making assumptions about how Amazon EBS and Google Persistent Disk work, but we want to do it in a more general way that is more supportable in the long term.) These are less sexy than things like reflinks, but the ROI is much easier to justify, both because the costs are lower (it's less work to develop, test, and qualify for enterprise deployment) and the benefits are much easier to quantify. Things like "I can save the cost of XX Full-time Software Engineers'salary over five years" are much easier to make for these sorts of performance improvement features.


              In contrast, reflinks are fun, but I haven't been able to find a customer willing to pay the development costs, or a company who thinks that their customers would be willing to buy more of their product if they added reflinks to ext4. This might sound horribly corporate, but there's a story about how the ZFS engineers started the project on the down lo, without asking permission from management or getting input from sales, and presented Sun with what was effectively a fiat accompli. Which might sound great,until you reflect that Sun ended up losing money until they had to sell themselves to another company, and effectively there is no longer much of an engineering organization supporting ZFS. Around the time that ZFS was announced, I participated in a company-wide investigation about whether it made businesss since to invest in file system features for AIX and Linux --- and the conclusion that we came to was, no, the ROI wasn't there and new file system features would not result in more customers buying IBM hardware, software, or systems. IBM may have fallen on hard times, but it's still around, and Sun isn't.

              Also around this time, representatives from multiple Linux companies came together to come up with a story of how Linux would compete with ZFS. It was at this meeting that the idea that btrfs would be the long-term answer, and ext4 would be the short-term solution that would supply support forthings like on-line resizing, 64-bit block numbers, and other things which traditional Legacy Unix OS's had that ext3 didn't. At that meeting, one of the things I was asked to do was to size what it would take to do a brand new file system. I did my researching, looking at how much effort was required to bring file systems's like IBM's GPFS and JFS, Digital's advfs, an estimate of what it took Sun to come up with ZFS, and to bring that file system to a fully enterprise ready production status. The answer I came up with was around 100 person years worth of effort, with one low-end estimate of 50 person years, and a high-end estimate of 200 person-years (but that was for GPFS, which was a cluster file system, and so a lot more complicated). I reported this findings to the meeting, and a certain senior engineer from Intel said, "No, don't tell the manager's that because they will never approve the project! Tell them that btrfs will be ready in 18 months." I'll let people decide when btrfs hit that "enterprise ready status", especially for those sexy new advanced features that were supposed to compete with ZFS, but I don't think it's controversial that it wasn't in 18 months. And even before Sun imploded, many of the companies who sent representatives to the meeting declined to actually contribute engineers to the btrfs effort, and that surely didn't help. But that was probably because companies are rational entities that are making their own ROI decisions, and funding a new file system didn't make as much sense as telling people that Linux would have an answer to ZFS.

              Looking back, time has proven that while ZFS had these really cool features, they weren't sufficient to cause most customers to decide to go with Solaris as compared to buying much cheaper x86 platforms and running Linux. And by the time Sun decided to try with OpenSolaris and Solaris x86 strategy, it was too little, to late. The network effects were huge, and the x86 strategy didn't have an answer to how a single company, Sun could pay the salaries for all of the super-talented engineer that worked on Solaris. Buying a $5,000 x86 server doesn't have much profit margin compared to a $100,000 SunFire E10k Sparc server which Sun billed as the "dot" in "dot Com".

              The bottom line is that engineering in the real world is all about tradeoffs, and business realities are part of that tradeoff. I make no apologies for the fact that I prefer to have food with my meals, and that I want to make enough money that I can retire some day. And that in turn means that I need to be comfortable understanding how I am deiivering at least 10x my compensation as value to my employer. If I can do that while still doing open source, and helping other companies make money so they are willing to contribute to ext4, well, that's part of the challenge and why I love working with Open Source.

              And going back to the Code of Conduct; it's not because of any kind of wussy liberal reasons that pretty much all of the mainline file system maintainers supported the CoC. It's because we need every single engineer who is willing to contribute to our project, and most of us have seen people who have declined to work on Linux and transferred to other operating systems (I know of one person who moved over to Windows who was a valued Linux kernel contributor at the IBM Linux Technology Center) or would work on internal projects but not anything that required interacting with LKML, because of the toxic environment of a few people on the mailing list. In some cases the fear was unfounded; for example Linus would yell at a senior developer who really should have known better and who in most cases Linus had met in person and they had a pre-existing relationship. The problem is that the newbies didn't know this and the got scared off --- "what if Linus were to humiliate me in public the way he did with Steve", not realizing that this in practice would't happen. This is why we have the CoC; it's not for us senior engineers, but to support the more junior engineers on our teams that we want to mentor to at some point, replace us when it's time to retire or we get hit by a bus or otherwise shuffle off this mortal coil.

              Remember the 50 to 100 person years of effort to make an enterprise ready file system. We need every single engineer we can get and many of us do extra work on our own time because we care. Getting a high quality file system is a team effort, and we need every single talented engineer we can get. Even if a single engineer is a super 10x programmer, if they end up scaring off a whole bunch of other engineers who might be working on testing or performance tuning, etc., it's just not worth it to enable someone who might be an asshole.

              Comment

              • nocturn9x
                Junior Member
                • Jan 2025
                • 2

                #37
                Originally posted by skeevy420 View Post

                Technically speaking, limiting speech and restricting what people can and can't do are conservative, right wing values. Codes of Conduct are only seen as left wing because they restrict hate speech and restricting hate speech upsets right wing people. Conservative, right wing people have no problem restricting supposed cuss words on television, regulating sexuality, limiting medical health and what doctors can do, praising the police state and mass deportation and imprisonment programs, limiting access to welfare and food, and more. That's what makes it hard to tell the difference between a right wing person and a racist regardless of what country they're from or the color of their skin -- because they back every freedom limiting initiative that effects everyone that's not themselves.

                It's fun watching y'all conservative right wingers get pissed off when you're forced to have to follow the same rules and regulations as everyone else. Oh my God, actual equality in practice. Oh the horror. Oh the humanity.

                If the Code of Conduct didn't punish y'all for saying words that the Phoronix Auto Moderator flags as hate speech y'all would love it because it's yet another tool to force your will on others. Sucks for them that they can't debate without calling people highly regarded ninjas.
                Had to make an account just for this one. I don't necessarily disagree with what you said (I do take issue with the usage of the world equality, I guess it depends exactly what you mean by it) but I think the problem here wasn't that conservatives got pissed off at some lefties, the problem is that Linus is the very reason why this CoC was ever put in place: he has always taken each and every opportunity to verbally attack everything and anything he doesn't like, and still no one bats an eye when he does so to this day. Such is the pain of a project having a benevolent dictator for life, I guess.

                Comment

                • Volta
                  Senior Member
                  • Apr 2019
                  • 2314

                  #38
                  Originally posted by amxfonseca View Post

                  This would be a worthy comment on Phononix, but unfortunately you are posting it on Moronix, so you are going to get all the hate.
                  You're one of the morons here. It's so called left wing that censorships everything. You won't tell me european union is right wing, will you?

                  P.S. categorizing by sides is stupid on its own, though. There are better ways to categorize people. You're just victims of manipulations.

                  Comment

                  • ahrs
                    Senior Member
                    • Apr 2021
                    • 589

                    #39
                    Originally posted by avis View Post

                    Stop using ext4 immediately. Rests in Theodore Ts'o's hands.
                    There's enough distributions using ext4 as their default filesystem that someone would step up if something were to happen to Theo. I don't know if I can say the same about Bcachefs. Do any distros even offer it as an option in their installer yet?

                    Comment

                    • koverstreet
                      Junior Member
                      • Jul 2016
                      • 18

                      #40
                      Originally posted by ahrs View Post

                      There's enough distributions using ext4 as their default filesystem that someone would step up if something were to happen to Theo. I don't know if I can say the same about Bcachefs. Do any distros even offer it as an option in their installer yet?
                      Only the very niche ones - I've been explicitly telling the distros to slow down on shipping bcachefs until the experimental label is off, which might (fingers crossed!) be six months away. We need at a minimum all critical bug reports resolved (plus some time with no new ones), additional repair torture testing, and a bit more on disk format work (some more minor backpointers stuff) before that can happen.

                      With anything this big staging the release is really important. I only want it in the hands of users who know what they're doing right now, I don't want average users getting burned.

                      Comment

                      Working...
                      X