Announcement

Collapse
No announcement yet.

The Latest Progress On Rust For The Linux Kernel

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by jacob View Post

    which wouldn't pass the borrow checker. But for the sake of the argument, let's assume that they do. On the first attempt to run this, RefCell's assertion will fail and they will have to fix it. Now of course, it is possible that this kind of code would be in some rarely used execution path and wouldn't fail in normal usage scenarios, that brings us back to the tests coverage problem. So obviously no, the current model is not perfect and does not do EVERYTHING that could possibly done statically. Yes, the borrow checker could be improved to model RefCell in simple cases like that. Is it worth all the effort though? That's the real question. If someone comes and volunteers to contribute to the compiler to implement specifically this, it's likely to be a decision between the (arguably meagre) benefit vs the cost of increased complexity and hardcoding library-specific rules within the compiler itself. But I don't think it's likely the core team itself would consider working on this anytime soon given the arguably much higher priority issues that are in greater demand too (generic associated types, improvements of const generics, further developments of the async support and ecosystem, etc.)
    One thing to add ontop of this is that its impossible for a turing complete type checker to ensure that every theoritical program is properly checked because that is essentially the halting problem (or more specially Gödel's incompleteness theorem)

    In other words, you will always find an example where the type checker at compile time is unable to prove that a correct program is correct, so a solution like RefCell will always have to exist.

    The best you can do is to limit the use of RefCell by making sure that the borrow checker can incrementally validate more and more correct programs but at one point you essentially hit diminishing returns and/or create other problems (such as increasing compile times). There is a strong argument that we have pretty much already hit that limit.

    And yeah, pointing out the flaws of RefCell by causing it to panic when the borrow checker at compile time can catch such errors is an asinine and misleading example, people use RefCell as a last resort. its like asInstanceOf/null in Scala or unsafePerformIO in Haskell, you just don't use it unless you have a really good reason.

    Originally posted by oiaohm View Post
    Yes the problem here that we have people making the rust compiler putting a false divide between what is compiler functionality and what is standard library functionality. This false divide makes them not see that the compiler should be checking bits of the standard libraries functionality to pick up human errors in coding before they cause end user problems.
    Actually this is the hallmarks of a well designed language, the more you couple the standard library with the compiler the less portable the compiler becomes and the harder it becomes for the compiler to reason about programs because now you have assumptions in the stdlib that aren't entirely correct for some given context being placed into the compiler (as a person who works with Java/Scala + JDK I have dealt with this many times).

    Also taking into account on what I said earlier, I don't know what bundling the standard library and compiler will achieve. Its not like you can break the laws of mathematics to magically make the borrow checker better (if your complaining that the compile time borrow checker is not good enough), and moving more of the borrow semantics into runtime is opposite of what Rust stands for from a design principle.
    Last edited by mdedetrich; 16 September 2021, 11:36 AM.

    Comment


    • Originally posted by jacob View Post
      This is exactly the kind of trivial case that can be handled statically. Obviously for the hell of me I can't see why would someone use RefCell here instead of the exactly equivalent code:

      Code:
      let mut c = 5;
      let m = &mut c;
      let b = &c;
      The answer is horrible simple. Person does not know better and using Refcell got rid of their build error and this was not replaced with a warning so they now believe they have done the right thing.

      Originally posted by jacob View Post
      which wouldn't pass the borrow checker. But for the sake of the argument, let's assume that they do. On the first attempt to run this, RefCell's assertion will fail and they will have to fix it. Now of course, it is possible that this kind of code would be in some rarely used execution path and wouldn't fail in normal usage scenarios, that brings us back to the tests coverage problem. So obviously no, the current model is not perfect and does not do EVERYTHING that could possibly done statically. Yes, the borrow checker could be improved to model RefCell in simple cases like that. Is it worth all the effort though? That's the real question. If someone comes and volunteers to contribute to the compiler to implement specifically this, it's likely to be a decision between the (arguably meagre) benefit vs the cost of increased complexity and hardcoding library-specific rules within the compiler itself. But I don't think it's likely the core team itself would consider working on this anytime soon given the arguably much higher priority issues that are in greater demand too (generic associated types, improvements of const generics, further developments of the async support and ecosystem, etc.)
      Please note I said at least a warning. Lets say rust had __attribute__() like gcc has. Items like RefCell in the standard could have a __attribute__() added to raise a warning. So you swap failure to build by using RefCell for warning that you need to validate this code at runtime. And by declaring what it is with a __attribute__ you could have the option to check it like a normal mut. That could be useful if you do have a runtime crash.

      Originally posted by ssokolow View Post
      That was a conscious decision. Rust was designed with the intent to work toward minimizing the coupling between the compiler and the standard library so that efforts to provide alternative standard libraries (eg. a klibc for the Linux kernel) won't be inherently crippled compared to the main one.
      Good in theory there is reason why this does not work out its human interaction with compliers.

      Originally posted by ssokolow View Post
      Checking bad uses of RefCell is allowed... it just belongs as a lint in Clippy, rather than as part of rustc.
      There is a problem with this logic. More and more lint functionality of C and C++ is being integrated into the compiler. Why developers are lazy. This is the reality if the compiler builds a lot of developers will not run test-suites and will not run lint tools. Yes why Linus of the Linux kernel wants -Werror on by default to reduce the amount of defective code landing on his desk.

      Comment


      • Originally posted by mdedetrich View Post
        Actually this is the hallmarks of a well designed language, the more you couple the standard library with the compiler the less portable the compiler becomes and the harder it becomes for the compiler to reason about programs because now you have assumptions in the stdlib that aren't entirely correct for some given context being placed into the compiler (as a person who works with Java/Scala + JDK I have dealt with this many times).
        Java and Scala don't have anything like __attribute__() this does come from another language this idea.

        Originally posted by mdedetrich View Post
        Also taking into account on what I said earlier, I don't know what bundling the standard library and compiler will achieve. Its not like you can break the laws of mathematics to magically make the borrow checker better (if your complaining that the compile time borrow checker is not good enough), and moving more of the borrow semantics into runtime is opposite of what Rust stands for from a design principle.
        This is a point people bring up to raise a warning does not mean you have to have always proved their is a problem. So having a __attribute__() that you could put on functions like RefCell so if they are used the compiler will raise a warning that you have runtime borrow checking in this program that need to validated because if its not this program could crash without warning. This stop some people swapping normal mut or Cell that will be checked for a RefCell. Because they got rid of a build failure and replaced it with a warning instead of nothing.

        Please note I did not say bundling standard library with compiler. There need to be way for compiler and standard library to work with each other. A way that the standard library functions can be marked that the compiler can provide sane warnings. And at times provide extra checking as in this function is like X function it usages should be the same Y static borrow checked function even that it runtime borrow checking.

        Java and Scala did attempt hard code this in the compiler. Where is java and Scala equal to the __attribute__() feature.

        mdedetrich basically there is a need for the standard library function decelerations to be able to provide the compiler with enough information to display valid warnings when user is possible doing something dangerous and at times to allow the compiler to perform deeper lint/static checks.

        mdedetrich think about it why can you use runtime borrow checking with rust and it does not raise a warning.

        Originally posted by mdedetrich View Post
        The best you can do is to limit the use of RefCell by making sure that the borrow checker can incrementally validate more and more correct programs but at one point you essentially hit diminishing returns and/or create other problems (such as increasing compile times). There is a strong argument that we have pretty much already hit that limit.
        Think about it how are you limiting the usage of RefCell if usage does not raise a warning. Making sure complier inform users when they are using runtime borrow checking would not result in running into the mathematical problem. If likes __attribute__() are applied correct code that uses runtime borrow checking it would be possible with a compiler flag to force checking when you have a problem or are able to take the extended compile times.

        Having the static and runtime borrow checking being able to be used as a combined solution.

        mdedetrich think about this at the moment you can only run the compiler you cannot look at the source code how would you be able to tell at the moment what programs used static borrow checking and what programs are using runtime borrow checking. This is a weakness of the rust compiler design and language of rust not allowing the standard language library and other libraries to provide compiler with more details to provide better developer information on what they just built.

        Comment


        • Originally posted by oiaohm View Post
          Please note I said at least a warning. Lets say rust had __attribute__() like gcc has.
          Rust has attributes and makes them even more of a first-class thing than that. The Rust equivalent to __attribute__((...)) is #[...] before the AST node in question or or #![...] as the first child of it and users are introduced to them much earlier and more readily than in C.

          For example, #[must_use] is an annotation you can apply to a type or function to get a warning if the return value is neither saved nor explicitly discarded with let _ = and stuff like #[derive(Debug, Clone)] is how you can ask the compiler to synthesize common functionality for structs and enums so long as their fields all implement said traits (interfaces).

          #[rustfmt::skip] is how you ask rustfmt not to touch the formatting of something when reformatting a file and #![forbid(unsafe_code)] at the top of the root file of a project is how you ask the Rust compiler to disallow unsafe use across the entire project without exceptions. (deny allows exceptions and allow/warn/deny/forbid are how you set the behaviour of "warnings", even though not all of them warn by default)

          There's even a standard way for procedural macros to hook in custom derives and, within what's fed to a procedural macro through any means, the macro can accept and process whatever attributes it likes. (That's how Serde and StructOpt provide very nice declarative APIs for parsing and serializing stuff.)

          It's also how the standard library achieves "collapse Option<NonNull<T>> into the same bit pattern as a nullable pointer to T" by using attributes to annotate when types have holes in their possible range that an Option's None value can be folded into.

          Originally posted by oiaohm View Post
          Items like RefCell in the standard could have a __attribute__() added to raise a warning. So you swap failure to build by using RefCell for warning that you need to validate this code at runtime.
          How is that not just "ask every legitimate user of RefCell to add a 'Yes, I'm not an idiot and I know what I'm doing. Disable the warning.' annotation to their projects"?

          I'm reminded of one of Raymond Chen's posts on his blog, The Old New Thing, where he's talking about the difficulty in designing UAC to be useful without causing "alert fatigue", where people just tune out and start clicking "allow" automatically.

          Originally posted by oiaohm View Post
          And by declaring what it is with a __attribute__ you could have the option to check it like a normal mut. That could be useful if you do have a runtime crash.
          Are you still misunderstanding that the point of RefCell is to be used in situations where the compiler can't "check it like a normal mut"?
          Last edited by ssokolow; 16 September 2021, 01:49 PM.

          Comment


          • Originally posted by ssokolow View Post
            How is that not just "ask every legitimate user of RefCell to add a 'Yes, I'm not an idiot and I know what I'm doing. Disable the warning.' annotation to their projects"?

            I'm reminded of one of Raymond Chen's posts on his blog, The Old New Thing, where he's talking about the difficulty in designing UAC to be useful without causing "alert fatigue", where people just tune out and start clicking "allow" automatically.
            To be correct having every legitimate user of RefCell add 'Yes, I'm not an idiot and I know what I'm doing. Disable the warning.' is a valid move.
            https://gcc.gnu.org/onlinedocs/gcc/D...c-Pragmas.html

            Gcc and LLVM have that functionality for C. Think about it when you use RefCell if you have to write large code because its high risk code it makes the safe code the more compact and simpler to read. This is about encouraging developers to use the safer route when possible,

            Yes you are right there is a balance between alert fatigue and alerts. Remember alert fatigue is not a reason not to have a fire alarm. Alert fatigue is not a reason to be missing means to generate a warning/error.

            It would be a option that you cannot use any runtime borrow checking unless you put in extra code to enable runtime borrow checking. Again this is about having the standard library and the compiler work with each other.

            From what you have said with the [...] you have the frame work at least partly their to implement proper warnings in case users attempt to use RefCell to fix a build error and don't know what they are really doing.

            Remember currently RefCell in lines of code is the same size as mut or equal that is safely processed by the compiler.

            Comment


            • Originally posted by oiaohm View Post
              The answer is horrible simple. Person does not know better and using Refcell got rid of their build error and this was not replaced with a warning so they now believe they have done the right thing.



              Please note I said at least a warning. Lets say rust had __attribute__() like gcc has. Items like RefCell in the standard could have a __attribute__() added to raise a warning. So you swap failure to build by using RefCell for warning that you need to validate this code at runtime. And by declaring what it is with a __attribute__ you could have the option to check it like a normal mut. That could be useful if you do have a runtime crash.
              This is a seriously contrived scenario. Basically you are supposing that the user is:
              • a beginner who struggles to get past the borrow checker even in this simplest of cases; but
              • at the same time, knows about RefCell and how to use it; and
              • understands that this is just equivalent to a mut& that he couldn't get to work in the first place, and can instruct the compiler about it.
              Really?

              Even leaving aside the total implausibility of the situation, let's say that you implement support for something like #[mut_equivalent] to be used with RefCell. What exactly is the point of that? In every case you could use it, you could just as well use the mut&, which is a better and more correct solution. Ultimately your linter would do nothing but print a warning "use mut instead" every time this annotation would be used. In fact mdedetrich's analogy above is exactly right: RefCell is like performUnsafeIO in Haskell, something to be used as a desperate last resort and probably only temporarily until the code is refactored. There is absolutely no point in having any linting for performUnsafeIO because in every case where it would apply, you should rather use the normal IO monad.


              Originally posted by .oiaohm View Post
              There is a problem with this logic. More and more lint functionality of C and C++ is being integrated into the compiler. Why developers are lazy. This is the reality if the compiler builds a lot of developers will not run test-suites and will not run lint tools. Yes why Linus of the Linux kernel wants -Werror on by default to reduce the amount of defective code landing on his desk.
              But that comparison is entirely flawed. Linting functionality added to C/C++ invovles things like strcpy() which should never be used, no ifs or buts (the only linting functionality here is "is it used at all? Then warning"), and things like printf() format strings, which are 1) used all the time and 2) trivial to check. Conversely, RefCell is 1) rarely used and 2) impossible to model statically except in irrelevant cases.

              Comment

              Working...
              X