Announcement

Collapse
No announcement yet.

The Issues With The Linux Kernel DRM, Continued

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #21
    Originally posted by airlied View Post
    fuck your an idiot, none of what you said is any way correct, what did you fucking do read some tea leaves to dig up this bullshit?

    Dave.

    No, it's about the fact that I expect to be pushed code that is WRITTEN AND TESTED BEFORE THE MERGE WINDOW.

    The merge window is not for writing new code. The code that gets merged should have been written two weeks ago already. The only new code that I want to see are actual regressions.

    I have been talking about this for YEARS now. It's not a new issue. I hate seeing patches sent to me while they are clearly still being discussed and developed. There's something seriously wrong there when that happens.

    Linus


    That and it seems that the DRM/DRI drivers are given quite a bit more free reign to break compatibility with older drivers and change their userspace interfaces. That's a big no-no the vast majority of the time.

    Which is fine. Doesn't bother me personally.

    Comment


    • #22
      My impression was that the DRM developers were following the same rules as everyone else, even though those rules result in a bigger drag on GPU development than on other devices because the hardware evolves so quickly.
      That's not something that is unique to video card drivers. It is probably a bigger issue for GPU then other things, because GPUs are still in such a state of flux, but it's still a issue for everybody.

      Linux kernel has always had a big problem getting driver fixes and improvements out to users in a timely basis. Say a new piece of hardware gets developed, whether it's a video card/wifi/video capture card or whatever. The device gets developed internally for a few months to a year or two and is released to the public. This is usually the first time Linux devs can get their hands on it.

      Then they hack together some drivers, get it out to users. Some users are intrepid enough to compile it themselves and test it. If the drivers are exceptionally difficult to compile as modules, like Wifi drivers in the past were, then you have to patch the kernel and this creates additional barriers to end users to test with.

      Meanwhile this device is out to market for months now. Windows users enjoyed the use of drivers from the first day it has been released and Linux users are still stuck with compiling code for relative untested drivers. Now the driver developers submit the code to kernel devs. If they are very lucky it will get accepted into the appropriate kernel tree. If they are unlucky it gets rejected outright and they have to go back and re-engineer the driver to suite the kernel developer's aesthetics if they have any hope of getting it merged.

      After a rewrite or two then the driver get it's code into a development or staging tree. Depending on the attention of the development tree owner, and issues with other drivers that may share code or touch the same interfaces, the driver may get submitted right away when the Linus merge window opens up, or it may not. It may languish for one or two release cycles after that.

      Meanwhile all this time the driver has been unavailable to the vast majority of Linux users. Finally the driver gets into the official tree. it gets accepted and into the vanilla kernel tree.

      End users STILL don't have access to it. Most end users still must wait for their distributions to pick up the kernel, play around with it, package it, and then have it work it's way through the distro's own code review and testing process.

      So all of this causes huge delays in the chain. For the new device out there it can take anywhere from six months to two years before it really reaches distros.

      That sort of thing is a real killer and a very huge and unpleasant side effect of the 'no stable internal ABI' that Linux has. It results in technically better solutions and much better tested code then your typical Windows release gets.

      But it's still extremely frustrating for users and driver developers.

      There are quite a few subsystems that went those this sort of hell for a long time. SATA support for Linux took years. Wifi support was a far worse nightmare which took a extremely long time to resolve and can still cause problems when new hardware gets released or new standards are created.

      Comment


      • #23
        Originally posted by drag View Post
        That sort of thing is a real killer and a very huge and unpleasant side effect of the 'no stable internal ABI' that Linux has. It results in technically better solutions and much better tested code then your typical Windows release gets.

        But it's still extremely frustrating for users and driver developers.

        There are quite a few subsystems that went those this sort of hell for a long time. SATA support for Linux took years. Wifi support was a far worse nightmare which took a extremely long time to resolve and can still cause problems when new hardware gets released or new standards are created.
        First you wrong, we never ever whatsoever broke the DRM api, we follow the same rules as others. If you think about KMS its different because kms has a major version bumps which means it's considered as different driver (i guess we could have build a different module name to make it more clear). Another example is the vblank patch that seeded this discusions, it grows the API but it's backward compatible ie old userspace will keep working unmodified.

        Second stable ABI is non sense go read all the mails about that they are full of actual fact and proper technical explanation why it's non sense.

        On the delay to get driver to the enduser fault is on the hw manufacturer, if they want their device supported they must work with the community on open source driver before the hw hit the market. Even without a stable ABI they still can provide a module to build against different kernel if they want a stop gap solution.

        So to sumup only thing i agree is the pain caused by absence of driver but i place the blame on the manufacturer who are the only one guilty here.

        Comment


        • #24
          Second stable ABI is non sense go read all the mails about that they are full of actual fact and proper technical explanation why it's non sense.
          I don't care about internal ABI. If the kernel devs don't want it then that is fine by me. They should know better then anybody.

          But there are consequences to decisions like that and slow time to market is one of them. This is core to the issue going on right now.

          Linux does not have a internal ABI that is non-existant, but a external API/ABI for the kernel is something that is supported and it is taken very seriously. The DRI protocol is part of the userspace API and as such would normally not be allowed to change.

          Probably the DRI folks should have free reign over whatever userspace interface they would like to use. Adding, subtracting, and changing interfaces as they see fit. However that would also mean that you couldn't use new kernels in older systems and it would create barriers for users testing other Linux features since it would break their userspace drivers.

          I am quite likely being a idiot in all of this, but that's fine. It doesn't really bother me too much.

          On the delay to get driver to the enduser fault is on the hw manufacturer, if they want their device supported they must work with the community on open source driver before the hw hit the market. Even without a stable ABI they still can provide a module to build against different kernel if they want a stop gap solution.
          The only way for manufacturers to support open source code ready immediately prior to release of the hardware to public is by either:

          A) doing the development internally by their own employees and keep the code secret until it's dumped in the kernel's lap shortly before the hardware goes to market
          B) or forcing the Linux developers to sign NDA's and get them to cooperate on suppressing access to code and internal documents prior to the release of the hardware. And they would have to be very careful about controlling information on next generation hardware development that would be going on during the entire process.

          Ideally the driver developers would work with the hardware engineers _during_development_ of the hardware. But that is not happening for a lot of reasons.

          Comment


          • #25
            First you wrong, we never ever whatsoever broke the DRM api, we follow the same rules as others.
            Ok. My apologies then.

            Comment


            • #26
              Linus just wants his job to be easy and painless. I've seen this pattern increasingly over the years; as the size of the kernel grows (or as Linus becomes more complacent), he pushes development further and further away from the things he touches. In a few years, will he want to only pull code into mainline that has been rigorously tested in a RHEL stable release for 7 years before merging it?

              The way I see it, the onus is not on Linus to provide bug-free kernel releases. That onus is on distributions, and other hackers and corporations who work downstream from the mainline kernel. Ordinary users don't go to www.kernel.org and download the latest kernel as casually as if they were updating Firefox. I know some people here on Phoronix do; but that is only about 1% of the user base (or 0.0000001% if you count all the Android devices out there nowadays).

              I wish the Linux kernel releases were simply time based snapshots of the state of the Linux kernel at a given time. Sure, some effort can be made to fix regressions and improve code quality as the release nears, so that each kernel version has some quality to it. But it is not the responsibility of mainline to ensure that their tagged releases are pristine. Because the direct consumers of mainline's work are software engineers who know how to do their own QA and fix their own bugs, not end-users.

              And indeed, if you look at the current practices of big organizations such as Google's work on Android; Canonical's work on Ubuntu; and Red Hat's work on RHEL; you see clearly that each company has a very large kernel team that invests thousands of man hours in testing, cherry picking fixes, and backporting features to build up their own kernel build to the degree of stability that they require. So if the stability of the mainline point releases were to falter, it wouldn't affect their practices at all. Even with the level of rigor we have today, the kernel point releases are untested enough that entire teams of engineers are needed to test and debug the kernel to the point that it is suitable for production use.

              And besides, who is to dictate that every distribution must always take the latest kernel version and use that as the basis for their distro? If a "bad" point release comes out with a lot of difficult to fix bugs, they can just use another version. That's why RHEL released a version based on their investment in 2.6.32 despite the fact that, at the time of release, 2.6.37 was available. If Linus' code quality standards are so high, why isn't Red Hat feeling free to ship RHEL6 using 2.6.37? Why is my Android 2.2 device based on a heavily hacked version of 2.6.32?

              I think the mainline Linux kernel is much more useful as a common starting point for software engineers to grab the latest code and start to meld it to their specific use case. It is a common development place to discuss changes and fixes, and a standardized point of distribution for the sources. But it isn't an end-user product.

              Linus is unintentionally hurting the development of Linux by trying to make less work for himself. This perceived "work" only exists because he has such a strong feeling of obligation that the point releases need to be perfect, so that a hapless user can download it and install it into a distribution shipped with a kernel 6 months old without any problems. This just isn't a necessary thing.

              I say the DRM developers should start developing in earnest out-of-tree and not worry about Linus' rules. We can't expect Linus to change his mind on this, but we can ask the DRM developers to create an environment for themselves where they feel that they can make steady progress on their code, without feeling like they are being painted into a corner by a dictator who doesn't understand their unique issues.

              And to the extent that I expect to run production devices on Linux, I will continue to expect my distribution to be the first line of defense, the people who convert the mainline Linux development code into a production-worthy product. Linus thinks that's his job, but it just isn't and shouldn't be.

              Comment


              • #27
                Reply-Edit (1 minute edit limit, I'm looking at you!):

                Just to clarify: I would really like to see the current DRM maintainers in mainline just resign from that position, and start maintaining DRM out-of-tree. Without a maintainer, DRM would eventually get dropped from mainline. This would solve Linus' problems, as he would have no code to bitch about and his life would be easy, just living in happy happy TCP stack world where nothing ever goes wrong. And it would solve the DRM developers' problems, as they could release "raw" code with arbitrary amounts of ABI breakage in the interest of providing the best performance and features to users. Everyone would win.

                Well... it would put more burden on distributions to decide what version of DRM to pull into their releases, but oh well -- I like to see the distributions doing something useful for a change. They certainly seem to have the manpower -- the stable kernel teams for all the big name distros have quite a few paid, full-time developers.

                Comment


                • #28
                  Originally posted by drag View Post
                  I don't care about internal ABI. If the kernel devs don't want it then that is fine by me. They should know better then anybody.

                  But there are consequences to decisions like that and slow time to market is one of them. This is core to the issue going on right now.
                  As i said non stable ABI doesn't forbid hw manufacturer to work on their code behind closed door and drop a working module that build against several different version kernel tree (98% of the times there would be no change btw several version of a kernel for set of API) on hw release. So no stable ABI doesn't hurt.

                  Stable ABI would only allow the write code once, dump it and let it rot full of bug attitude. If we had stable ABI manufacturer would be doing closed source driver and we would have even worse hw support than windows. By having floating ABI we force hw manufacturer to consider open source and upstreaming to ease their burden. Note that there is lot more reasons behind stable ABI that are more technical.

                  Originally posted by drag View Post
                  The only way for manufacturers to support open source code ready immediately prior to release of the hardware to public is by either:

                  A) doing the development internally by their own employees and keep the code secret until it's dumped in the kernel's lap shortly before the hardware goes to market
                  B) or forcing the Linux developers to sign NDA's and get them to cooperate on suppressing access to code and internal documents prior to the release of the hardware. And they would have to be very careful about controlling information on next generation hardware development that would be going on during the entire process.

                  Ideally the driver developers would work with the hardware engineers _during_development_ of the hardware. But that is not happening for a lot of reasons.
                  A & B is already happening for AMD & Intel (weither about CPU or GPU or chipset) it's likely not perfect, and there is still lag on it but they are working on this. The GPU side is bit different as we have been playing catchup while doing major change all over the stack. Writting a GL stack is not a 6month project.

                  Comment


                  • #29
                    yeah. I've used hardware that development was maintained outside of the kernel for various reasons.

                    It's not something that is a good thing.

                    Comment


                    • #30
                      As i said non stable ABI doesn't forbid hw manufacturer to work on their code behind closed door and drop a working module that build against several different version kernel tree (98% of the times there would be no change btw several version of a kernel for set of API) on hw release. So no stable ABI doesn't hurt.
                      That's not what I was saying. You have my points mixed up. My apologies for not being clearer.

                      If you had a 'stable ABI' for the kernel then what could theoretically happens is you do your development on the DRM drivers and then users can just download the kernel modules from your site and use them immediately. Right?

                      Right now development happens in private trees, then the development gets loaded into the staging trees, then gets loaded into the vanilla kernel, then gets loaded into the distro trees... were it sits and waits for distros to do a major release and THEN the users get easy access to it.

                      Each step along the way creates delays and is another barrier between the drivers and end users.

                      Don't construe this as meaning that I want a stable internal ABI. Between Linux and other Unix-like systems Linux has far and away the best hardware support and best drivers for the widest amount of systems. Solaris, OS X, FreeBSD, etc etc. Linux has them all beat and I think that 'no stable internal ABI' has a lot to do with it.

                      A & B is already happening for AMD & Intel (weither about CPU or GPU or chipset) it's likely not perfect, and there is still lag on it but they are working on this. The GPU side is bit different as we have been playing catchup while doing major change all over the stack. Writting a GL stack is not a 6month project.
                      No, obviously it's a major undertaking. I am actually quite happy at the pace that ATI is gaining support for new hardware with open source drivers.

                      I am using R600 right now and it's pretty damn good considering everything going on.

                      Comment

                      Working...
                      X