Gosh, Our Test Farm Already Finds Big Regression

[Michael]

Ugh, there's more regressions... Well, all of the data will be publicly out next week -- and from there on out in a real-time stream -- so more can join in on the testing and bisecting fun if you wish.

[kraftman]

Cool.

For the bashers: It's their software, it's their money, it's their time. They release when and the way they like it.

Yeah, what a usefull information...

Those who comment (bash in your opinion) probably want to have this regression fixed and probably wondered if Phoronix will help with this. And it seems they will. Thanks.

[talvik]

Cool.

Cool.

For the bashers: It's their software, it's their money, it's their time. They release when and the way they like it.

[Michael]

This delay is just not so the public interface can be finished up but to also ensure the issue can be reproduced on other platforms and that the commit that's isolated is indeed correct. This will hopefully be all completed by Monday so it's really not much of a delay.

[yoshi314]

i have to applaud phoronix for kernel test farm. this will definitely be helpful. also, we need more people concerned with linux desktop performance, as contrary to most kernel developers (as proven by Con Kolivas' experience).

the only good reason for not disclosing a performance regression is that it has to verified in more test environments.

and this probably isn't the place to say it, but I do think Phoronix needs to have better wording in the articles to avoid that kind of impression - if I might exaggerate a little, less of "look aren't we great, here's sensationalist news" and more "here's something interesting".

the one in this article sounds to me this way - "here's some interesting performance regression in the linux kernel, but you have to wait one week when our next great product comes out to learn about it." . there's pretty much a marketing decision.


phoronix should generally disclose the regression as quickly as possible. if it's real - kernel test farm will already prove a valuable tool, regardless of its release date - today, tomorrow or maybe even never (Coverity is extremely useful, but it was never released to the public)

[mirv]

waiting and not disclosing security bugs is bad.

performance regressions are not as critical, but i think not disclosing them during merge window is a fatal mistake, because it might be very difficult to revert the problematic merge.

publicity stunt.

I'm normally quiet about this sort of thing, but I agree with the publicity stunt opinion. And this probably isn't the place to say it, but I do think Phoronix needs to have better wording in the articles to avoid that kind of impression - if I might exaggerate a little, less of "look aren't we great, here's sensationalist news" and more "here's something interesting".
Don't get me wrong, a solid automated testing framework to discover regressions would be very useful - just the article reads too much like an opinionated blog.
Of course, feel free to disagree with me, I won't be offended.

[mirza]

Compier-related suggestions

Here are some suggestins:

1) It would be nice to have tests on both gcc and icc (and LLVM?). Gcc-generated code quality messure over time, on real world SW, is missing.

2) Tests for different versions of gcc, inluding ancient (3.x).

3) Tests for default compiler settings vs. CPU type specific options.

[yoshi314]

And if it's a regression, then a patch fixing it would still be accepted outside the merge window, and basically anytime until the final release.

waiting and not disclosing security bugs is bad.

performance regressions are not as critical, but i think not disclosing them during merge window is a fatal mistake, because it might be very difficult to revert the problematic merge.

I don't even know what to call this move. It's not selfish because there's not really anything to gain/

publicity stunt.

[[Knuckles]]

And you are holding off disclosing this because???? Seriously, what do you have to gain by not submitting this. Every day you wait brings the release date for .33 closer, and makes it less likely that there will be time to create/test patches.

I don't even know what to call this move. It's not selfish because there's not really anything to gain/

.33 should still be 3 months away or something -- the merge window is still open and -rc1 hasn't even come out yet.
And if it's a regression, then a patch fixing it would still be accepted outside the merge window, and basically anytime until the final release.

[jbrown96]

And you are holding off disclosing this because???? Seriously, what do you have to gain by not submitting this. Every day you wait brings the release date for .33 closer, and makes it less likely that there will be time to create/test patches.

I don't even know what to call this move. It's not selfish because there's not really anything to gain/