Linux VGEM Driver Rewritten In Rust Sent Out For Review

[coder]

Actually, before libpthread got rolled into the same .so file, glibc would switch between atomic and non-atomic refcounting for shared_ptr based on whether the libpthread symbols had been linked into the binary.

If true, that's merely a subset of the behavior changes that occur from failing to use -pthread. I'd argue it's not a std::shared_ptr<> issue, but rather libc overhang from a time when multithreading was the exception far more than the norm.

now that every language including C has threading primitives, would be to forget about pthreads completely and instead have the compiler and the language's std lib use the low-level clone() system call and its friends directly.

Interoperability with other APIs and tools benefits from using a common, underlying representation. pthreads also has certain useful features that would need to be duplicated by any replacement implementation. For instance, you can enable race-detection on any mutex class which provides access to the pthreads representation.

[STiAT]

I find that surprising since Rust's ownership system and other features like pattern matching makes things pretty easy to follow. I used to write in C# and would put in tremendous effort to basically make an ad-hoc runtime ownership system so I could track resources and execution paths easier. With Rust, it's baked right in. I don't even need to write tests or run the program, it's usually pretty obvious how things work just from how the code is laid out and annotated. Also helps that it has a culture of documenting absolutely everything.

I've just implemented two small projects in Rust yet, one on embedded devices, which is my plant watering system which I moved from Python to Rust, and a middleware which was supposed to be small and only had few features and calls written in Go (expected 1k calls / min), which somehow grew unexpectedly to quite some calls (~50-75k/min) while the code base remained small (still few features). After I reimplemented it showed less resource usage and better performance.

So I'm pretty new to Rust and just didn't put the time in yet to easily be able to read and write code. Don't get me wrong, I see the benefits, but reading isn't as easy as it is in other languages, especially the Attributes stuff quite still is ... interesting to me, where I think it's a feature available and commonly used in C#, and still a feature I do not understand why it is needed, and I couldn't find any good explanation on why it makes sense. Highly reminds me Macros with a scope.

[ssokolow]

especially the Attributes stuff quite still is ... interesting to me, where I think it's a feature available and commonly used in C#, and still a feature I do not understand why it is needed, and I couldn't find any good explanation on why it makes sense. Highly reminds me Macros with a scope.

It's not just C#. Java has them and calls them annotations. Python has them but calls them decorators. Under the same name Python uses, they're a stage 2 proposal in JavaScript and TypeScript has an experimental implementation of them. I stopped using Java before it gained them and my only experience with C# is helping my brother solve very specific problems in Unity, but, in Rust and Python, they're basically an alternative syntax for procedural macros. (In Python, putting @my_decorator above my_func or my_class is literally just syntactic sugar for putting my_func = my_decorator(my_func) below it.)

They're needed for two reasons:

1. They can be used in positions that the more ordinary macro syntax isn't allowed because it'd violate rules about ordinary macro syntax producing a certain kind of well-formed sub-tree of the AST.
2. They allow applying procedural macros to things without throwing out all assumptions about what token streams are valid within the procedural macro body, which makes dev tooling easier. (Inside a traditional macro body, the only thing you can be certain of without consulting the macro is that it must have successfully gotten through the Rust tokenizer.)

That said, in Rust, there's a distinction between attributes and attribute macros. For something like this...

#[serde_as]
#[derive(Deserialize)]
pub struct ChannelMeta<'a> {
   // TODO: savefile.js indicates there's "position", "topic", and "nsfw" if ServerType::Server

   #[serde(borrow)]
   name: Cow<'a, str>,

   /// Index into [`DumpMeta::servers`]
   server: DiscordServerIndex, // TODO: Resolve
}
​

... #[serde_as] is an attribute macro, #[derive(Deserialize)] is a derive macro, and #[serde(borrow)] is an attribute that's not a macro itself but, rather, gets parsed by the code invoked by #[derive(Deserialize)].

[STiAT]

They're needed for two reasons:

1. They can be used in positions that the more ordinary macro syntax isn't allowed because it'd violate rules about ordinary macro syntax producing a certain kind of well-formed sub-tree of the AST.
2. They allow applying procedural macros to things without throwing out all assumptions about what token streams are valid within the procedural macro body, which makes dev tooling easier. (Inside a traditional macro body, the only thing you can be certain of without consulting the macro is that it must have successfully gotten through the Rust tokenizer.)

That said, in Rust, there's a distinction between attributes and attribute macros. For something like this...

So basically, we are talking about procedural macros in function like macros and derives.

I think I do not understand the when and where, and why we need that split, so the limitations on this. Since if I use function like macros, basically the same can be done but without providing a function scope, if I got that right.

From a design perspective I find it interesting. What if you require two different Deserialize derived from two different crates? That can happen pretty fast dealing with data streams.

But well, as I said, I just do not have enough experience to actually come to proper conclusions or to have enough experience to know how to properly deal with situations like that. That's what comes when I work a few month exclusively on a project only utilizing Rust, and I was not there yet - sadly.

And my background is only C, C++ and Go (the past few years), so I'm at a loss with Java, C# or JavaScript (and I really grew to dislike Python so I only use it when really necessary), I can read and write in those languages, but I'm not even close to somebody really knowing the features those languages really provide.

[ssokolow]

From a design perspective I find it interesting. What if you require two different Deserialize derived from two different crates? That can happen pretty fast dealing with data streams.

Same considerations as if your data needs to back two different implementations of the same abstract class in C++. You'd probably have two different structs and a conversion between them.

Think of structs with Serialize/Deserialize on them as akin to XML or JSON schemas. You have one per serialized representation.

As for the "two different crates" part, if you can't add derive support to both using a Cargo feature (compile-time flag), Serde has #[serde(remote = "...")] to work around the orphan rule and allow you to implement a serialization/deserialization for someone else's type.

[STiAT]

Same considerations as if your data needs to back two different implementations of the same abstract class in C++. You'd probably have two different structs and a conversion between them.

Think of structs with Serialize/Deserialize on them as akin to XML or JSON schemas. You have one per serialized representation.

As for the "two different crates" part, if you can't add derive support to both using a Cargo feature (compile-time flag), Serde has #[serde(remote = "...")] to work around the orphan rule and allow you to implement a serialization/deserialization for someone else's type.

That's interesting, I thought of the two abstract classes before, but I didn't know about the #[serde(remote = "...")] . That's what I mean by I just have way too little experience in Rust yet to answer all those questions. I'll probably need a bigger project to work on for some time to get to know its features, the reasons behind them and their usage better. Same happened in other languages - with projects came experience, and with experience you learned you made mistakes or misunderstood features / didn't use it properly.

[coder]

I'm arguing that if a car is built of cardboard and leaks fuel, then seatbelts and airbags might be nice to have but don't change the fact that that car is a deathtrap.

To use your analogy, the car's body got upgraded to fiberglass and its fuel lines are now steel-reinforced. It's still a car, it still burns combustible fuel, but it's a lot safer than before. Not to mention that it handles better.

Which is exactly the reason why they are. Having a "clue what you're doing" is a strawman.

It's not, and I can see why you don't understand that, because developing lint tools tells you all the ways something can go wrong, without the accompanying data about how often it happens in practice.

In a 20 MLoC project developed over many years by dozens or hundreds of people, no-one has a clue what someone else did ten years ago in an unrelated part of the codebase but that still subtly and indirectly interacts with the one you're developing,

First, let's acknowledge that your own personal experience is just that. It doesn't reflect all projects, nor can you know how well it represents any other given project.

Second, not all projects have the same level of disorganization or mismanagement. In languages like C and C++, you need to be explicit about ownership-transfer and scope, when bare pointers or references are being passed across an API. This is either accomplished by the use of clear naming conventions and/or documentation. If you guys had no such policies in place, then shame on you. Better yet would be to use shared_ptr, etc., but that's not always an option (especially in older projects).

Finally, the primary purpose of a software architecture is precisely to minimize pairwise interactions between the component parts. Again, not all projects are well-architected.

How is that an argument?

The conclusion doesn't follow from the premise.

That is exactly how it happened indeed, and what does it show? That auto_ptr was a purported solution that didn't really work.

This entire line of argumentation is essentially a red herring. You've run out of arguments about modern C++, so now you're attacking its history. As I said before, we could go through the evolution of Rust and pick apart everything that didn't work and had to be revised. That's exactly what you're doing to C++. It proves nothing about modern C++. It's just cannon fodder for you to use in making C++ seem worse than it is.

a quarter of a century after the release of a language

What are we considering the "release of the language"? 1998 is only 13 years before 2011. And if Rust had been released, in its current form, back in 1998, it would've been virtually unusable. Programming language developments don't happen in a vacuum. They can't be decoupled from the hardware used to run the compilers.

I'm exposing my arguments and so far, you didn't show that any of them is factually wrong.

I said they're disingenuous, not wrong. You're over-representing nits and minor pitfalls as if they're cataclysmic. Your entire premise that modern C++ is no safer than legacy C++ was just a troll, so that you could spread more anti-C++ FUD.

The problem with race conditions affects mainly shared_ptr, as I described.

shared_ptr, itself, has no problem with race conditions. It's simply not a substitute for synchronization on interaction with whatever it owns. That's not a problem with shared_ptr.

unique_ptr's eternal problem is its non-uniqueness.

More like a theoretical problem. To be usable and reasonably safe, a language or library feature needn't be completely devoid of pitfalls.

Look at it this way: if you go back to using bare pointers as in C code, do you think double-free bugs + memory leaks would be more or less common? If we're being honest, they could pretty much only be more common. Ergo, unique_ptr improves safety, even if it's not the 100% guarantee you'd like it to be. That doesn't make it broken. That just makes it not fool-proof.

In Rust, there are two guarantees: you can't wrap a non-thread-safe object in an Arc and mutate it in an uncontrolled way across threads (it simply won't compile). You can't also ever pass the cheaper, non-atomic Rc it between threads (it won't compile). And, secondly, unless your Arc wraps something atomic, like a Cell or a Mutex, then get_mut() is fallible and your code won't compile if it doesn't take care of that fact.

If I follow your same line of reasoning, I can say that because Rust doesn't prevent deadlocks, that makes it no safer than C++. To be clear, that's not what I believe, but that's essentially the argument you're making about modern C++.

Your argument is akin to claiming that Fortran is perfectly safe, you just need to ...

Nope. I never said modern C++ is "perfectly safe". I just said it's safer than legacy C++.

May I ask then, why do you believe it's a good idea to keep using C++ when better alternatives are available?

Because a rewrite is lots of work and can't always be justified for mature projects.

I'm obviously specifically asking about greenfield projects where the re-use of or integration with an existing ecosystem is not an issue (those are legitimate cases for using C++ of course).

Well, then you're putting words in my mouth, because I didn't say that.

There there, dude, no need to get personal.

Nice victim-blaming, troll. If you want to have a civilized discussion, don't argue under false pretenses and to further an agenda at whatever cost.

If I baited you into a discussion, only to use you as a foil to exaggerate a handful of Rust's failures and shortcomings, and then concluded with a knee-slapping non-sequitur about it being mathematically proven fact that Rust is no safer than C++, you'd be rightly suspicious of my grasp of mathematical proofs.