Canonical Releases Upstart 1.6

[Delgarde]

Writing good tests is hard.

Yep, and measuring how good they are is even harder. That's why I don't put much stock in talk about how good a project's coverage is - I've seen too many of them where the tests are just plain worthless. I've yet to see a good execution of it, but I'm very fond of the idea of mutation testing - systems for making subtle changes to the code (e.g changing a plus to a minus, or an && to an ||), and seeing how effective the tests are at catching them.

I'd especially not put too much faith in projects claiming 100% coverage. My experience is that to get to perfect coverage, the test code is going to such contortions in testing the rarest edge cases, that the test is far more complicated than the code it's testing and probably contains at least one significant bug.

[Ericg]

I'm not sure what this "after the next LTS release" crap is since Ubuntu just had an LTS and they don't seem afraid to implement major changes in their LTS releases anyway (e.g. pulseaudio in Hardy).

The assumption is it would take a few releases to migrate things over from upstart to systemd and by the time it was done it'd be release window for the next LTS and Canonical has gotten more serious about "LTS MEANS STABILITY!!!" since Hardy.

[DanL]

Ubuntu's parent (Debian) is still using sysvinit and Ubuntu has already addressed the shortcomings that are important to them with upstart. systemd has more features, but those obviously don't outweigh the negatives in Canonical's eyes. Ubuntu won't bother with systemd until it gets a lot more testing (and Ubuntu probably doesn't care enough to use their limited resources to contribute to that effort). I don't fault them for that, but I'm not one of those people that flies off the handle every time there's a Canonical article.

I'm not sure what this "after the next LTS release" crap is since Ubuntu just had an LTS and they don't seem afraid to implement major changes in their LTS releases anyway (e.g. pulseaudio in Hardy).

[ryao]

Umm, no. The size of the test suite says *nothing* about the quality of the code - what matters is whether the tests actually do their job in validating the code works correctly I've worked on projects with near-perfect code coverage, only to find that many of the tests just asserted what the code did, bugs and all.

Writing good tests is hard.

[Malizor]

Umm, no. The size of the test suite says *nothing* about the quality of the code - what matters is whether the tests actually do their job in validating the code works correctly I've worked on projects with near-perfect code coverage, only to find that many of the tests just asserted what the code did, bugs and all.

You are right, of course.

It's hard to objectively judge the quality of a project. This is a combination of aspects.

On top of tests suites sizes and user bases, we could also consider the number of open bugs (excluding feature requests).
In this case, Upstart has 62 and Systemd has 85 (feel free to correct me if I messed too much with filters).

Still not a formal proof of anything, but (to me) all clues seem to confirm that Upstart is more mature than Systemd...

[Delgarde]

My point is: as Upstart has a big test suite (and probably a bigger user base than Systemd) it is "proven" to work.

Umm, no. The size of the test suite says *nothing* about the quality of the code - what matters is whether the tests actually do their job in validating the code works correctly I've worked on projects with near-perfect code coverage, only to find that many of the tests just asserted what the code did, bugs and all.

[tjaalton]

Dont confuse yourself. I said the last activity regarding tests in systemd was from sep so stop blabbering about "not evovling". That has nothing to do with the link.

BTW I find it very amusing to look at ohloh. Canonicals own testing people seems to like systemd better than than their own homebrewed shit. Martin Pitt have shown more activity on systemd than every non-lead upstart maintainer COMBINED. When your own test engineers goes with the competition then you truly are fucked. Canonical failed to build a community and they failed to gather the interest from their on testing crew.

that's because udev was merged with systemd, so every update to udev is now shown as an update to systemd..

[funkSTAR]

You source "it's better since September" by quoting a post from April?

Dont confuse yourself. I said the last activity regarding tests in systemd was from sep so stop blabbering about "not evovling". That has nothing to do with the link.

BTW I find it very amusing to look at ohloh. Canonicals own testing people seems to like systemd better than than their own homebrewed shit. Martin Pitt have shown more activity on systemd than every non-lead upstart maintainer COMBINED. When your own test engineers goes with the competition then you truly are fucked. Canonical failed to build a community and they failed to gather the interest from their on testing crew.

[Malizor]

Well it acually did in september if you look closer. And it is outpacing Upstart.
https://plus.google.com/u/0/11554768...ts/JCDko6rkic5

BTW who decided test cases is the gate keeper for init system stability? The past holds no record of this. Sure it is nice to have but ?t is not THAT essential. Having a pissing contest about who can add most test cases wont help anything.

You source "it's better since September" by quoting a post from April?
(which is false otherwise, look at the sixth comment...)

My point is: as Upstart has a big test suite (and probably a bigger user base than Systemd) it is "proven" to work. You can trust new versions stability.
And yes, init unbreakable stability is important out there.

[funkSTAR]

This was a question. You know, the things that can be answered without the need to be rude.

By quickly looking at the source code, it seems to me that systemd did not really evolve on test coverage. That's a shame.

Well it acually did in september if you look closer. And it is outpacing Upstart.
https://plus.google.com/u/0/11554768...ts/JCDko6rkic5

BTW who decided test cases is the gate keeper for init system stability? The past holds no record of this. Sure it is nice to have but ?t is not THAT essential. Having a pissing contest about who can add most test cases wont help anything.