Nebulet: A Rust Microkernel Running WebAssembly In Ring 0

[lachlansneff]

Nebulet uses only a single 64-bit address space on x86_64.

[Weasel]

So all the programs that run under this OS are threads?

At ring 0 every program can read every other program memory. The VM is the only thing preventing but behaviour. But in case of a bug...BOOM!

Yes, it's exactly like MS-DOS, roughly speaking here. (I'm talking about process-separation behavior, basically meaning there's none)

[lachlansneff]

Process isolation is ensured by webassembly only being able to represent 32bit pointers. If the os allocates a contiguous region of mapped and unmapped memory that is 4gb (well, 8gb really), we can avoid including any bounds-checking in the generated machine code.

[oiaohm]

So all the programs that run under this OS are threads?

Current prototype you would say is threads. Protection is not implemented. You have to start somewhere.

At ring 0 every program can read every other program memory. The VM is the only thing preventing but behaviour. But in case of a bug...BOOM!

https://arxiv.org/pdf/1706.07257.pdf

To block ROP attacks at ring 0 you have hardware added to different cpus to check code flow and memory access flow. Ring 0 means direct access to hardware in all cases. Depend on cpu design of security features and implemented features in operating system depends if something at ring 0 can read everything in memory or not.

Gen-Z hardware is expected to start 2019/2020 this hardware has memory security built in. Per cpu chip/cpu core based on how the chip is made. If you are using per core memory security on Gen-Z having ring 0 in core does not grant permission to read or write all memory. You can only read and write to the memory allocated to that core. Yes if memory is allocated to a core read only it cannot write to it.. Memory security and ring levels are two different things in a Gen-Z system. Yes a gen-z system can have x86 cpus. Most people don't consider that memory security and cpu core can be completely split. Part of Gen-z is taking the MMU back out the CPU and implementing security at the MMU. So no a VM is not the only thing that can prevent the ring 0 from being able to read and write where ever it likes. Current hardware that we mostly have is fairly limited on memory security causing the ring 0 god power problem the design of hardware for next few years has very high memory security so stripping a lot of god powers off ring 0.

So you must change the value of CR3 often, flush the TLBs, etc... Humm it is starting looking like context switch to me!

NUMA Each process could be give to a core and each core at ring 0 can run a different memory map this has been done in some NUMA real-time operating systems to improve performance in ring 0 where drivers are given own memory map and own core. Please note having a cost of context switching to start/switch/stop process and not having any context switch cost to access hardware is still a saving on overhead.

Basically there are differences between ring 3 and ring 0 other than just the memory access. Device access is a big one. Also some of the modern cpus designs their hardware protections against rop and other attacks at ring 0 have reduced the difference between ring 0 and ring 3 from security point of view. Saying CR3 says you are thinking only current day x86 cpus.

Of course the difference reduction by hardware improvements does not mean you want to run absolutely random items at ring 0 but items like ring 0 being able to access every bit memory comes false in the newer hardware designs planned. The prototype could be someone working on getting the basic working before Gen-Z and equal hardware lands. Once Gen-z and equal hardware lands implement memory security in memory security and not care about ring levels as much.

Please note gen-z bus system also provides hardware access security so being ring 0 in a cpu and the gen-z bus saying you cpu is totally untrusted the program running on that cpu is not accessing anything. Bus security is something of a new idea. Bus security has advantage over rings. Bus security does not care about the internal state of cpu cores so allowing you to have a system with mixed cpu types. So you could have a computer with risc-v, arm and x86 cpus inside all sharing the same video cards, harddrives and other features. Of course this is going to make operating system design a little more tricky. Moving from ring to bus security is a very big change. This up coming change means ring 0 is not going to remain as powerful as it has been historically.

[lachlansneff]

As the writer of nebulet, I would like to say that many of you seem to have an incorrect sense of how far nebulet has progressed.

[oiaohm]

As the writer of nebulet, I would like to say that many of you seem to have an incorrect sense of how far nebulet has progressed.

I see it as a prototype, I am more looking forwards to if the idea has a functional chance. I don't see the problems of being in ring 0 as a long term problem short term yes there are some issues due to how cpu and motherboards have been designed up until now. I am not really expecting nebulet to be production ready before we have Gen-Z and other bus secured hardware turning up. Ie end of 2019 start of 2020 at the earliest .

Bus secured hardware is quite a change and it means we do need to rethink on bus secured hardware if ring levels in cpu make any sense at all or should that just all be bus security. The next few years are going to be interesting to say the least.

[pabloski]

NUMA Each process could be give to a core and each core at ring 0 can run a different memory map this has been done in some NUMA real-time operating systems to improve performance in ring 0 where drivers are given own memory map and own core. Please note having a cost of context switching to start/switch/stop process and not having any context switch cost to access hardware is still a saving on overhead.

So you have only one process per core. In a 4 core CPU you can only run 4 processes? Is this what you are suggesting? Otherwise you have a bunch of processes per core and you must change some registers ( like CR3 ) to support page tables.

[Paradox Ethereal]

I suggested something similar on my webpage aswell: https://nyt.cloud/forumdisplay.php?fid=3

Irix + BSD + Amiga = BSDmodX, for Fair Money Indie Dream OS.

Peace.

[oiaohm]

So you have only one process per core. In a 4 core CPU you can only run 4 processes? Is this what you are suggesting? Otherwise you have a bunch of processes per core and you must change some registers ( like CR3 ) to support page tables.

Depends on your core. You are thinking way too much Intel where you page tables are a cpu feature. Power processor runs 8 threads per core at the same time each of those threads could be given a different memory id that gen-z bus could enforce.

https://en.wikipedia.org/wiki/Barrel_processor

Barrel processor design allows for thousands of threads/virtual cores per core. There are risc-v prototypes with this feature.

The need to change registers and page tables to have a bunch of processes per core is limitation of common in usage cpu designs but those are not the only way to design cpu.

Bus security ideas come out of some experimental soc where each core does not have a clue about page tables or rings to keep the cores as small as possible. Yes when you want to fit 4096 cores in a processor socket you optimise a few things for reducing silicon size then question came could they put the security back someone notice that a lot could be put back at the mmu and cache level with memory/bus ids without increasing core size as much as having a cores that knew about page tables/rings.

So 2 to 3 year out the problem you are referring to might not be problem at all as this depends on what way future hardware goes. So the multi process problem does not have to be solved this early on in development thinking by my guess when it could be ready for production usage the problem may not exist. Being able to run 1 thread per core on existing hardware using NUMA separation would be a slow way to simulate where future hardware is possibly going.

Security done by memory management units/bus is a very different beast to common registers, rings and page tables designs.

[unixfan2001]

Here's hoping some of the research and development of Nebulet can be rolled into Cervus.
As much as I like the idea of new microkernels, Cervus is more interesting, IMO. At least for my use case.