The Latest Progress On Rust For The Linux Kernel

[binarybanana]

How allocation failure is handled is a purely library level thing in Rust, so worst case it can be signaled the same way as in C, but since it's Rust it will look like Option<*mut usize [u8]> (or some "Byte" type instead of u8, perhaps for non-8 bit byte arch support if that is a thing). Constructors might then perhaps return Result<T, AllocFailure> or some such. It's unfortunate that std is lacking here, just like it lacks custom allocator support for collections, but that doesn't affect how this can be handled in the kernel.

I'm not sure what role overcommit plays in all this, I guess that isn't even really a thing in kernel space? But yes, if you want allocation to ever actually fail you have to do it like Windows and require as least as much RAM+swap as has been allocated, even if it's untouched by the process, which opens up its own can of worms, like programs OOMing even if half your RAM is free. This makes Windows totally unusable without tons of swap. I had CyberPunk 2077 crash on me every other hour until I realized that I needed more swap even though RAM was still far from exhausted, plus the swap file wasn't even being filled, which made the error in the Windows event viewer pretty puzzling, indeed. Until I remembered that Windows doesn't do overcommit... If you absolutely *need* utmost reliability you don't allocation at runtime at all, like safety critical and most embedded systems. I don't really see the point of some awkward middle-ground like on Windows, but I guess some do.

[ultimA]

On various platforms (Linux included), you don't really have any choice, because the kernel defaults to overcommit semantics to compensate for applications that waste a ton of memory on Windows by mapping much more memory than they actually write to.

That means that, even if you're handling malloc failure perfectly well, malloc will return success but, when you try to write to that memory, the kernel may suddenly realize it's overpromised what memory is available and have to kill something after all the branches for handling allocation failure in the software have already been told it was successful.

This more of a theoretical issue than a real one. When an application requests memory from an allocator such as malloc, the allocator will rarely make any requests to the underlying OS, as it requests memory in batches of pages from the OS, and most times the allocator just returns memory from memory areas that it has already reserved a "long" time ago and have been written to. And even if the application is unlucky enough so that its current request just caused a new page reservation from the OS, the allocator will still have written the page for its internal bookkeeping before the app gets the new pointer returned.

[ssokolow]

This more of a theoretical issue than a real one. When an application requests memory from an allocator such as malloc, the allocator will rarely make any requests to the underlying OS, as it requests memory in batches of pages from the OS, and most times the allocator just returns memory from memory areas that it has already reserved a "long" time ago and have been written to. And even if the application is unlucky enough so that its current request just caused a new page reservation from the OS, the allocator will still have written the page for its internal bookkeeping before the app gets the new pointer returned.

Not necessarily. Nothing says the allocator needs to store metadata in every page allocated. For example, if you request 50 times the size of a page worth of contiguous memory (eg. to store a chunk of file contents), the allocator probably isn't going to write to every page returned.

[oiaohm]

No its not, check the source code of the Rustc compiler please, I already linked the actual compiler documentation previously.

That is not the define of the rust Language. The source code of the compiler is not a language define. Rust borrow checker is define in the rust language documentation and does include RefCell. Just because Rustc is a this stage still a incomplete implementation is not reason for stating the language defined bit wrong.

[mdedetrich]

That is not the define of the rust Language. The source code of the compiler is not a language define. Rust borrow checker is define in the rust language documentation and does include RefCell. Just because Rustc is a this stage still a incomplete implementation is not reason for stating the language defined bit wrong.

And the documentation for the compiler said the borrow checker is a compile time phase in the compiler. The documentation for RefCell says that it can satisfy borrow semantics at runtime (along with other things that has nothing to do with a borrow checker, as said previously its also a reference counter)

Try again

This more of a theoretical issue than a real one. When an application requests memory from an allocator such as malloc, the allocator will rarely make any requests to the underlying OS, as it requests memory in batches of pages from the OS, and most times the allocator just returns memory from memory areas that it has already reserved a "long" time ago and have been written to. And even if the application is unlucky enough so that its current request just caused a new page reservation from the OS, the allocator will still have written the page for its internal bookkeeping before the app gets the new pointer returned.

The point is there is a reason why the behavior of panic'ing/crashing when you are no longer able to allocate memory is a default. In practice, time and time again for typical applications/libraries its shown that this is the most reliable thing you can do. There are of course exceptions (such as the kernel itself and programs such as databases) but otherwise you should really crash (just after printing any relevant state, if possible)

You also need to distinguish between exceptions and business logic errors, no one is saying that an app should crash because you put text into a field that is expecting digits as input validation but there is a certain class of errors (such as being unable to allocation memory) where there isn't any reasonable behavior that an application can do.

Typically when these cases happens its considered a bug and you need to resolve it

[oleid]

That is not the define of the rust Language. The source code of the compiler is not a language define. Rust borrow checker is define in the rust language documentation and does include RefCell.

Wrong, rust is defined by its RFC process and the RFC repository. And certainly for parts which don't have an RFC by the source code.
Some of the links presented in this thread link to the "official book". That is written by third party and was donated to the community.

[cl333r]

Saying it doesn't make it so. The borrow checker is what the Rust developers say it is, what the compiler source code says it is and what the Rust community says it is. You could decide to call the libc the "C preprocessor" or you could decide to call New York "Paris", it wouldn't be the case. But at least you have successfully recognised the word "borrow" and jumped to a wrong conclusion, congratulations. That's still more intelligence than you've otherwise shown in this discussion.

Saying it doesn't make it so. The borrow checker is at the core of Rust development, it's Rust way to enforce the borrowing rules at compile and runtime.

[cl333r]

Just drop it. You have proven to be a clueless moron who knows nothing about Rust, concurrency or software safety and you are full of shit on each of these topics. mdedetrich was mistaken about RefCell but s/he is not hell bent on crass ignorance combined with unwarranted condescension. Unlike you.

It works both ways my little funky piglet, so just drop it. You have proven to be a clueless moron who knows nothing about Rust, concurrency or software safety and you are full of shit on each of these topics. mdedetrich was mistaken about RefCell but s/he is not hell bent on crass ignorance combined with unwarranted condescension. Unlike you.

[ssokolow]

Saying it doesn't make it so. The borrow checker is at the core of Rust development, it's Rust way to enforce the borrowing rules at compile and runtime.

borrow semantics are at the core of Rust development. The borrow checker, which is a compiler pass, is just the means by which they're enforced 99% of the time. Thus why people tend to conflate them.

[oiaohm]

And the documentation for the compiler said the borrow checker is a compile time phase in the compiler. The documentation for RefCell says that it can satisfy borrow semantics at runtime (along with other things that has nothing to do with a borrow checker, as said previously its also a reference counter)

Wrong, rust is defined by its RFC process and the RFC repository. And certainly for parts which don't have an RFC by the source code.
Some of the links presented in this thread link to the "official book". That is written by third party and was donated to the community.

RefCell.

https://rust-lang.github.io/rfcs/1660-try-borrow.html

oleid RefCell is written into the RFC as part of the borrow checking system.

mdedetrich the reality here the borrow checker part of the rust compiler has not caught up with what is written in the rust RFC that define the language. The borrow checker in the compiler should be checking the RefCell and other parts of runtime borrow checking as well for stupid errors. Ok down graded to warnings not failure to build.

The compiler is not the language define. RFC and the book written by a proper author basing what he is writting of RFC state clearly you have runtime borrow checking.

The reality is the rust compiler has a weakness that does need to be fixed. And its a weakness that means rustc compiler does not properly conform to what is written in the RFC of the language. Of course mdedetrich that flawed code of the compiler you have been trying to use to argue your point with me.

Like it or not the rust RFC that defines the language defines that you have runtime borrow checking and build time borrow checking. Big problem these don't properly overlap in the compiler. The build time borrow checker should be checking the runtime borrow checking as much as possible to catch programmer errors. Currently rustc is not attempting to check the rust runtime borrow checking this is a compiler flaw not a language flaw.