Systemd Looking At A Future With More Varlink & Less D-Bus For IPC

[varikonniemi]

Interesting that this is the third attempt to move away from dbus by the systemd gang. First it was kdbus. Then bus1. Then a hotfix was made in dbus-broker. And now Varlink.

Imagine the shithole we would be in if kdbus was accepted by kernel developers and the shit design of dbus was cemented into the kernel!

[energyman]

of course now that dbus is stable, well understood and around for years, systemd is going with something else. Why? Poettering. That is why.

[pabloski]

The open source world often likes to turn things upside down.

Yeah this is why I have turned my computing world upside down and have migrated to FreeBSD. And let's be clear, just because Open and Net aren't so good on (1) performance the former, (2) modern pc hardware support the latter. In a different world I would be using Haiku right now!

However I have real grown pissed of Poettering schenanigans and half-backed solutions, starting with the non deterministic systemd and ending with the crap that is flatpak and similar solutions.

[pabloski]

I don't know why an IPC mechanism couldn't support format negotiation of some sort.

this will come in 5-10 years, after leaving behind a half baked Varlink, Poettering will make another great slideshow to explain us that Varlink lacks format negotation, so he is releasing Varlinknegotiated as the new super duper ( and every distro must adopt it asap ) solution

[Weasel]

IPC generally needs several context switches, dont think writing/parsing json creates a new problem there.

[...]

i am more concerned with needing to search for nul bytes to find the end of the message.

lol..

[Almindor]

My first knee jerk reaction was "not JSON again". But then I thought about the problem space and realized JSON is perfect here. This is not a performance critical high-bandwidth transfer problem space. Having messages be this readable is a huge bonus in itself.

[DumbFsck]

My first knee jerk reaction was "not JSON again". But then I thought about the problem space and realized JSON is perfect here. This is not a performance critical high-bandwidth transfer problem space. Having messages be this readable is a huge bonus in itself.

Also, I will say, as a former json hater, that nowadays json really isn't that bad in general, due to how advanced the tooling has gotten.


The, few posts, above linked simd json is but an example.


I don't care about "web native" as a "principle" - bit being "web native" means there are millions of devs interacting with it every day, so there is much incentive to make it "better" (even if in some cases the efforts may be misguided and make things worse)

[reavertm]

zeromq+protobuf, done. Or instead of protobuf something that doesn't require two encoding passes (first one to count encoded size).
What does web native have to do with systemd? Is systemd becoming distrubuted operating system?

[coder]

if you're doing binary encoding, you want to look more at something like capnproto/protobufs/thrift/apache arrow, or more likely something like androids binder, so you can send real objects which was kinda the point of dbus being so complicated,

It's strongly-typed (or it can be), with even a notion of structs, arrays, and of course they can contain each other. The payload specifiers are defined here:

https://dbus.freedesktop.org/doc/api/html/dbus-protocol_8h_source.html

but it kinda was a total mess, and shouldn't have been anywhere near systemd to begin with.

systemd didn't just use dbus, they actually re-implemented it, presumably to avoid certain things they didn't like about libdbus. One thing I'll say for their sd-bus implementation is that it includes built-in support for the introspection interfaces, which makes it an absolute dream when using something like Python.

It means you can interact with DBus services by making remote method calls against them just like interacting with a local object. The python library automatically knows what parameters to expect and how to interpret the result. This is exactly what you want, for simple IPC uses (i.e. where you're just making synchronous method calls). I'd go as far as to say that if that was the only experience you had with DBus, you'd wonder what all the fuss is about!

I was looking into getting networkd or systemd events information out of dbus and it's a frustratiing mess, both because systemd documentation is terrible, but also because of dbus itself.

When I looked at it, several years ago, the sd-bus documentation was in poor enough shape that I had to dive into the code, on a couple occasions. In one case, it seemed the only answer to be found anywhere on the internet was by looking at one of the unit tests.

There's also a weird behavior I noticed, where sd-bus wants you to unref objects from the same thread that allocated them, which just seems to me like ref-counting done wrong. I suppose if unref'ing something potentially involves communication, then perhaps you want it to be synchronized with the protocol state machine, but I still think that's a pretty weak argument.

One last thing I really appreciated about DBus is that it's connection-oriented and supports disconnect notifications. DBus calls these "watches". They're a pain to implement properly, but they enable you to bind certain resources to a session. That's been really useful, in some of the services I've used DBus to implement.

[coder]

It also has an extremely limited type system: no integers (only floats), no precision, and maps can only have strings as keys. The Varlink FAQ's answer about integer precision is ... it's your problem.

It seems you misread the part about integers. Quoting from the link you provided:

Why are there no unsigned integers, and why are there no integer sizes specified?

Like JSON, varlink is intentionally underspecified in this regard. The implementation should use integers large enough to carry the signed integer value, and return an error if it cannot handle the received number.

So, it doesn't support unsigned ints, which makes sense if there's not also a specified range. IMO, the main place where unsigned integral data types are truly defensible is for bit masks, in which case you also need to know the range.

It's pretty unfortunate they just threw up their hands and took no stance on the minimum requirements for parsers' integer support. I guess you can safely assume int32 is supported, but perhaps truly portable code shouldn't expect peers to support int64?

JSON has no limitation foe integer sizes, you can use 128bit integers if you want.

But you have to know the peer supports that, which makes such an approach unsuitable for a generic service API that should be usable by an unbounded set of peers and varlink implementations.