We Can Automatically Find Functional Regressions Too

[phoronix]

Phoronix: We Can Automatically Find Functional Regressions Too

Last October we reported on new capabilities within the Phoronix Test Suite that allowed performance regressions to be located within any code-base by leveraging our open-source testing framework with the git-bisect command to automatically traverse a tree in a binary manner until the Phoronix Test Suite spots the regression-causing commit. As an example we tracked down a regression in the mainline Linux kernel with the EXT4 file-system without any manual intervention. Now not only can we automatically track down any performance regressions, but the Phoronix Test Suite can also help you spot any functional regressions...

http://www.phoronix.com/vr.php?view=ODM1Mw

[Timmmm]

Great stuff. I think the automatic testing work you're doing for linux is badly needed and unappreciated.

However, (and this is very off-topic), do you think you could tone down the advertising on your site? I just had to install adblock because I physically couldn't read this article with the flashing something2media ad to the side. The roll-over links are pretty annoying too. Static banners are fine.

Also, 90% of the real links in the article just go to a generic search (engadget do it a lot too), which is pretty useless. I assume you're doing it to increase your pagerank, but I'm pretty sure google doesn't count intra-site links.

Keep up the good work.

[Wyatt]

Oh hey, you reported the bug! Thanks!

[curaga]

Good stuff on the reporting.

Though, git bisect was already scriptable; how does using PTS for this change things?

[Michael]

Though, git bisect was already scriptable; how does using PTS for this change things?

PTS makes it very easy to just plug-in the test conditions and then to deploy a performance or functional regression test in a turn-key manner that's fully automated and reproducible.

[kernelOfTruth]

Great stuff. I think the automatic testing work you're doing for linux is badly needed and unappreciated.

However, (and this is very off-topic), do you think you could tone down the advertising on your site? I just had to install adblock because I physically couldn't read this article with the flashing something2media ad to the side. The roll-over links are pretty annoying too. Static banners are fine.

Also, 90% of the real links in the article just go to a generic search (engadget do it a lot too), which is pretty useless. I assume you're doing it to increase your pagerank, but I'm pretty sure google doesn't count intra-site links.

Keep up the good work.

please think of the (potential) epileptics among your readers (I'm not)


and thanks for adding a very useful feature & reporting it

[allquixotic]

Finding functional regressions is even more useful than finding performance regressions. Functional regressions are through-and-through defects, whereas performance is extremely situational.

OTOH, coming up with a true/false script for determining whether a defect is present can prove challenging. Unless you provide some helpers along those lines, it's basically another open programming environment prompting the developer to come up with an automatic way to determine if defects are present. Making that determination is often left up to test cases written in the native programming language of the project.

[hanglekuk]

But where is the bug? (follow the bugreport).
I guess some scripting to check if the latest libs are used would be useful too.

Waiting for PTS to catch some real bugs. Wonder what the revised title of this article would be...

[misiu_mp]

You dealt with it professionally. Good work.

[FireBurn]

Finally a decent article where you did something useful with the PTS!