Hi!
Some comments to help understanding what Tapper really is:
Tapper is exactly not yet another test or benchmark suite. It is a
test infrastructure which among other things allows to write or
embed test suites.
In that the focus is much larger, much more targeting an enterprise QA
life cycle than just being a test suite.
Tapper allows gluing together components based on existing
standards. One central idea for that is the standard Test Anything
Protocol (http://testanything.org) which allows language agnostic
generation of test results.
In order to run a Tapper client it is not neccessary to have a
Tapper server. You can run a TAP emitting testsuite with the standard
command "prove" that comes with every Linux system, like
Code:
prove ./tap-emitting-testsuite.sh
need a server. And then it's nothing more than just
Code:
./tap-emitting-testsuite.sh | netcat $tapperserver $port
Please see this presentation
to understand how any complex results, that are even allowed to
structurally change over time, can be tracked and evaluated.
It provides a query language that combines a database independent SQL
abstraction and a variant of XPath all in one concise API without the
need of client side dependencies. The point of that is to allow
test result forensics in ways you don't know in advance.
The opensourced version already ships with a wrapper for the autotest
client - that immediately provides you access to about 90(!) test
suites. For that the OSRC contributed TAP support to the autotest
project. See
autotest mailing list for details.
And Tapper is actually used with a Phoronix test suite wrapper at the
OSRC. For phoronix it would be most practical to be embedded into
autotest to be useful to more test communities. The Tapper-TestSuite-AutoTest
would then just pick it up automatically.
Tapper only implements features that were not available in 2008 when
they were needed, like virtualization setups for stress testing
(inclusive windows test suites in the guests), or when available
solutions were not enough, like the scheduler.
The automation layer provides a built-in installer to set up
virtualization scenarios but others can be plugged in easily. That is
actually done for kickstart/autoyast based setups and for simnow
testing.
On top of that the scheduler allows multiplexing machine pools with
any freely definable use-cases. It is built to schedule "too many
use-cases" on "not enough machines" based on bandwidths.
Setting up a Tapper instance for tracking test results is easy as all
dependencies are resolved by standard CPAN installation. The
non-trivial setup of an automation infrastructure with virtualization
support is additionally supported with the Tapper-Starterkit.
To complete a QA life cycle Tapper can schedule dedicated testplans
that belong to tasks defined by a planning software like
TaskJuggler. The results are reported back to TaskJuggler and
therefore covered in the project status pages.
The layers are strongly decoupled and all interfaces are very
lightweight, to allow glueing more things together in the future.
I hope I could clarify on some points.
Kind regards,
Steffen
Leave a comment: