Announcement

Collapse
No announcement yet.

Rust-Written Redox OS Closer To Self-Hosting

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by man-walking View Post
    I'm a fan of Rust but I wonder how do RedoxOS people dial with the continuous changes in the language and/or libraries, how do they keep aligned such big codebase ??
    Because there aren't any continuous changes in the language or libraries. We do often implement features we need in external libraries though, and update our Cargo.toml file to point to our latest commit for a temporary period of time until it gets mainlined though. And there's no need to deal with a change until we are ready to, as we all use semantic versioning in the Rust community. Occasionally, we will go through and update our Cargo.toml files, and if a breaking change did happen to occur, it's an easy fix due to the great compiler error messages. And now that Redox is an official target shipping within the Rust standard library, maintenance is a breeze.

    For the longest time, I had Ion shell exclusive to nightly Rust. Never had any problems with nightly compilers. As new features were added to the nightlies, I would just start using those new features wherever they are useful. Today, however, Ion builds on stable Rust -- hard requirement on at least Rust 1.19. It's great that Rust is continuing to see new features, as many of the features it gains are quite useful in writing more maintainable code in the long term.

    Comment


    • #12
      Originally posted by mmstick View Post
      Redox OS's Ion Shell is also pretty amazing, even on Linux / BSDs. It has a ton of features that you won't find in any other shells, and executes scripts 30-50% faster than Dash. Supports a simpler shell syntax that has the ability to configure 24-bit colors, string/array methods to eliminate the need for external commands for text manipulation, the ability to load external plugins to add more namespaces and methods, using functions within pipelines, arrays as separate first class citizens rather than conglomerating them with strings, and a number of useful builtin commands -- such as regex-based string match evaluation with the matches builtin.

      Code:
      echo ${color::bold,red,whitefg}Bold red text with a white background.${c::reset}
      echo ${color::0xF90}24-bit orange color${c::0x0090F0}24-bit blue color
      echo $method1($method2(input, arg), arg)
      for character in @graphemes("string"); echo $character; end
      let file_as_array = [ @lines($read("file.txt")) ]
      if matches Foo '([A-Z])\w+'; echo true; else echo false; end
      function1 | function2 arg arg | builtin | external
      > configure 24-bit colors

      Surely you could use a function instead of that special syntax. Is it possible to write functions to handle those ${fn::arg} calls?

      Comment


      • #13
        Originally posted by thechef View Post
        Why a microkernel when you're crash-safe at compile time? I understand that stable interfaces can be reasonable if you're not Linux, but if "micro-kernel" means address space isolation then I don't get it. If it just means _semantic_ microkernel as in compile-time isolation, stable API or software-isolation as in singularityOS then okay
        Rust's memory safety is an entirely separate thing than the security and features provided by having your kernel running as either a micro or hybrid kernel. It is still wise to develop secure software using secure methods, and a memory safe language to take care of the lower-level issues like out of bounds array accesses. We also have to consider that not all of the code running on Redox is going to be written in Rust. We may someday find, much later down the road, GPL'd Linux drivers being used by Redox's micro kernel. Easiest path to hardware support, honestly.

        As microkernel's are small, there is much less to maintain, and therefore much less that can go wrong. By not having drivers running in the kernel space, as they do in Linux, you avoid having to worry about a bug in a driver causing an attacker to elevate their privileges to root, by using the kernel itself to perform actions on the host that they would otherwise not have permission to do. It also makes it easier to, as you mention, recover from a fatal error in a driver. If your graphics driver panics, the kernel can just reload that driver. It won't take the kernel down with it.

        Comment


        • #14
          Originally posted by microcode View Post

          > configure 24-bit colors

          Surely you could use a function instead of that special syntax.
          It's not special syntax in Ion. Ion supports namespaces, which are denoted with a pair of colons. Currently, there exists two namespaces: color, and env. c is also shorthand for the color namespace, so there's no need to write the full namespace. The value that follows is the variable/constant/function that is being called from that namespace. Due to the nature of how ANSI codes are handled, the color namespace supports commas for specifying multiple values to grab from the inner tables when piecing together the complete ANSI code sequence.

          We also support an official git plugin, which when installed, adds a new git namespace, and supports three accessible variables: branch, untracked_count, and modified_count. The command,
          Code:
          echo ${git::branch}
          , would simply execute the branch function from within the git library, and get the result of that function back as a string, if possible. I may also be adding some support so that plugins may do similar actions as the color namespace, if their index function specifies a certain keyword.

          For functions that can accept arguments, you should look into methods. That is their purpose:

          Code:
          $function(input, arg1 arg2)
          The idea is that $function designates to the shell that we are going to execute a string method of that name, where a string method returns a string. The input element can either be a reference to a string variable, or an expression. Methods may treat array expressions differently than string expressions. Finally, the `arg` values will control the behavior of the method. Example:

          Code:
          echo $len(stringvar)
          echo $len(@arrayvar)
          echo $len($(cmd))
          echo $len([one two three])
          echo $len("string")
          We can tell that an array is being passed into a function if there is an expression that begins with either '@' or '['. Everything else is a string.

          Is it possible to write functions to handle those ${fn::arg} calls?
          If you mean being able to control the values within a braced string expansion, that's not supported at the moment, but it will be soon.

          Comment


          • #15
            Originally posted by thechef View Post
            Why a microkernel when you're crash-safe at compile time? I understand that stable interfaces can be reasonable if you're not Linux, but if "micro-kernel" means address space isolation then I don't get it. If it just means _semantic_ microkernel as in compile-time isolation, stable API or software-isolation as in singularityOS then okay
            As other people said, Rust programs can crash, and because of unsafe modules, they can (much less frequently than C/C++ programs usually) corrupt memory.
            But even if memory corruption were impossible, it's always possible to leak resources, and process isolation is a well established way to be able to clean up resources when you're done.

            Comment


            • #16
              Originally posted by mmstick View Post

              Rust's memory safety is an entirely separate thing than the security and features provided by having your kernel running as either a micro or hybrid kernel. It is still wise to develop secure software using secure methods, and a memory safe language to take care of the lower-level issues like out of bounds array accesses. We also have to consider that not all of the code running on Redox is going to be written in Rust. We may someday find, much later down the road, GPL'd Linux drivers being used by Redox's micro kernel. Easiest path to hardware support, honestly.

              As microkernel's are small, there is much less to maintain, and therefore much less that can go wrong. By not having drivers running in the kernel space, as they do in Linux, you avoid having to worry about a bug in a driver causing an attacker to elevate their privileges to root, by using the kernel itself to perform actions on the host that they would otherwise not have permission to do. It also makes it easier to, as you mention, recover from a fatal error in a driver. If your graphics driver panics, the kernel can just reload that driver. It won't take the kernel down with it.
              Practically I agree, but I in the long run I would prefer that the isolation mechanics of a micro-kernel merge with the mechanics of a compiler so that the compromise between security and performance is eliminated and also that security mechanics are consolidated in general. Rust is a nice example of a language where security and performance team up instead. Rust is faster, because it is also more secure. It introduces standard ways to handle security critical things that are traditionally left to the programmer. Those standard ways are highly optimized in one place in the compiler and therefore faster.

              Java went the route of being a compiler that has become a little bit of a kernel (remember its former name "Object Application Kernel"). The VM has many mechanics of an operating system (e.g. Garbage Collector, memory safety - it can isolate threads from each other). SingularityOS is basically the same experiment repeated but this time explicitly to replace operating system kernel-semantics. It has no address space isolation between processes. Software Isolation only.

              I don't like those approaches, because they are not eliminating the compromise between security and performance (regardless of their other negative effects that they can't execute unmanaged code). So can't we make a shift to compile time (and in turn use the compiler more often at runtime to compensate that)?

              I would rather like to see a classic operating system kernel to inherit features of a compiler. Theoretically Linux could introduce a free-for-all system-call run_rust(String rustCode) and run that code in kernel mode. There is almost no issue with that, because the kernel-integrated rust-compiler (with some enhancements (proper hooks for killing/preempting the program) and restrictions (safe rust only)) could ensure it is safe.
              I could write a wayland-display server in rust and run it in kernel mode and safe a few context switches when buffers and input events are redirected between userspace applications and display and peripheral hardware. That's my wishlist :-)
              Last edited by thechef; 18 August 2017, 01:29 PM.

              Comment


              • #17
                Originally posted by thechef View Post
                Why a microkernel when you're crash-safe at compile time? I understand that stable interfaces can be reasonable if you're not Linux, but if "micro-kernel" means address space isolation then I don't get it. If it just means _semantic_ microkernel as in compile-time isolation, stable API or software-isolation as in singularityOS then okay
                You can corrupt memory regardless of language or compiler involved. Even things like ECC does not 100% protect you from random bit flips in the hardware level.

                Comment


                • #18
                  Originally posted by mmstick View Post
                  Redox OS's Ion Shell is also pretty amazing, even on Linux / BSDs. It has a ton of features that you won't find in any other shells, and executes scripts 30-50% faster than Dash. Supports a simpler shell syntax that has the ability to configure 24-bit colors, string/array methods to eliminate the need for external commands for text manipulation, the ability to load external plugins to add more namespaces and methods, using functions within pipelines, arrays as separate first class citizens rather than conglomerating them with strings, and a number of useful builtin commands -- such as regex-based string match evaluation with the matches builtin.

                  Code:
                  echo ${color::bold,red,whitefg}Bold red text with a white background.${c::reset}
                  echo ${color::0xF90}24-bit orange color${c::0x0090F0}24-bit blue color
                  echo $method1($method2(input, arg), arg)
                  for character in @graphemes("string"); echo $character; end
                  let file_as_array = [ @lines($read("file.txt")) ]
                  if matches Foo '([A-Z])\w+'; echo true; else echo false; end
                  function1 | function2 arg arg | builtin | external
                  Wow that sounds really nice!

                  Comment


                  • #19
                    Originally posted by thechef View Post
                    I would rather like to see a classic operating system kernel to inherit features of a compiler. Theoretically Linux could introduce a free-for-all system-call run_rust(String rustCode) and run that code in kernel mode. There is almost no issue with that, because the kernel-integrated rust-compiler (with some enhancements (proper hooks for killing/preempting the program) and restrictions (safe rust only)) could ensure it is safe.
                    I could write a wayland-display server in rust and run it in kernel mode and safe a few context switches when buffers and input events are redirected between userspace applications and display and peripheral hardware. That's my wishlist :-)
                    That's an interesting idea; I think it should be possible with Rust as long as unsafe code is banned and the accessible functions are carefully considered (e.g. make sure untrusted code has no way to halt the kernel).

                    It would probably be a fairly large project, with various subtle issues to address. One issue is that the Rust compiler takes relatively long; perhaps it could use Rust's MIR (mid-level intermediate representation) as a bytecode; I don't know if that would allow appropriate safety guarantees (and it probably isn't a stable interface?).

                    Of course, it means adding a lot of code to the kernel, including LLVM.

                    Comment


                    • #20
                      Originally posted by mmstick View Post
                      a simpler shell syntax that has the ability to configure 24-bit colors, string/array methods to eliminate the need for external commands for text manipulation, the ability to load external plugins to add more namespaces and methods, using functions within pipelines, arrays as separate first class citizens rather than conglomerating them with strings, and a number of useful builtin commands -- such as regex-based string match evaluation with the matches builtin
                      Meh. How is it better than fish (except that fish is slow)?

                      I think fish did the right thing with "every variable is a proper array, strings are just 1-element arrays".
                      Last edited by andreano; 18 August 2017, 05:14 PM.

                      Comment

                      Working...
                      X