Announcement

Collapse
No announcement yet.

Defining The Phoronix Test Suite

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • phoronix
    started a topic Defining The Phoronix Test Suite

    Defining The Phoronix Test Suite

    Phoronix: Defining The Phoronix Test Suite

    It's going on four years that we have been providing benchmarks under Linux at Phoronix, and through this time the process has evolved as more applications have come about that gauge Linux system performance and meet our criteria for use in hardware reviews. While our internal process is continually refined, we have received requests for creating a Linux benchmarking manifest so that interested parties are able to easily reproduce our tests for comparative purposes and know everything that's involved. With that said, we are now working to establish the Phoronix Test Suite that will consist both of detailed documentation defining our hardware benchmarking protocols and free software that automates much of this work.

    http://www.phoronix.com/vr.php?view=11813

  • chem
    replied
    Originally posted by Michael View Post
    chem,

    I really wouldn't mind porting it to Mac OS X, but right now I have no Mac setups at all... Therefore inhibiting that work. Perhaps when I have the time and extra resources I'll pick up a Mac and start that work.
    Thanks for the quick reply. If you ever need someone to test a release on OS X 10.5 (Intel), just send me a message. Good luck!

    Leave a comment:


  • Michael
    replied
    chem,

    I really wouldn't mind porting it to Mac OS X, but right now I have no Mac setups at all... Therefore inhibiting that work. Perhaps when I have the time and extra resources I'll pick up a Mac and start that work.

    Leave a comment:


  • chem
    replied
    Originally posted by Michael View Post
    To address this question of multi-platform compatibility, the Phoronix Test Suite software is compatible with Solaris and I'd anticipate it even working on the *BSDs.

    However, the Phoronix Test Suite specification will be specifically designed and tested for Linux. Sure, you can apply the same testing practices to Windows or any other OS for that matter, but the chosen benchmarks and accompanying documentation will be all geared for Linux testing.
    I am currently installing the 0.51 PTS on my Fedora 8 desktop. However, I would love to compare its results (on either the "universe" suite or even just a subset of tests) to my MacBook Pro laptop, running OS X 10.5. As OS X is certified Unix '03, BSD-based, and has a variety of GNU/Linux programs available via Macports or Fink, I imagine it would be not too much trouble to get PTS working (at least in part!) on OS X.

    I think this would be an incredibly valuable contribution. You could even compare benchmark scores on the same hardware (macbook pro, in my case) booted to Linux vs OS X. Currently, I do not know of a good suite to do this.

    While I understand that Phoronix makes their money from Linux, I hope that Phoronix reconsiders and at least makes a perfunctory effort to make their PTS compatible with OS X 10.5. It may even be a wise business or PR move.

    Thanks.

    Leave a comment:


  • SavageX
    replied
    I'd just like to add that it makes sense to run the (Nexuiz) benchmark more than once to make sure everything is cached and whatnot - but I guess this is already done.

    Leave a comment:


  • Michael
    replied
    Doh, I did +benchmark instead of -benchmark... Yeah, it's now working. Thanks Kano.

    Leave a comment:


  • Kano
    replied
    It works with nexus 2.4 - 100%. I am using my sid backport on etch:

    ii nexuiz 2.4-1 A fast-paced 3D first-person shooter

    Leave a comment:


  • Michael
    replied
    Originally posted by Kano View Post
    nexuiz -benchmark demos/demo1

    result shown or in

    .nexuiz/data/benchmark.log

    PS: Als long as you run free games - best those which are packaged in Debian I could even create a live cd for testing... nexuiz (without music) + gl2benchmark fits on one cd iso.
    The benchmark argument doesn't work in Nexuiz 2.4. The timedemo argument works, but it doesn't quit automatically. Aside from that, the profile is working.

    Matthew Tippett has already recommend creating a Live CD/DVD, so maybe your offer will be used once the GUI for PTS is in place, etc.

    Leave a comment:


  • Kano
    replied
    nexuiz -benchmark demos/demo1

    result shown or in

    .nexuiz/data/benchmark.log

    PS: Als long as you run free games - best those which are packaged in Debian I could even create a live cd for testing... nexuiz (without music) + gl2benchmark fits on one cd iso.

    Leave a comment:


  • Michael
    replied
    Originally posted by SavageX View Post
    As basically all Quake-engine based games it has the nice timedemo command. Try e.g. "timedemo demos/demo1". I think it should even work directly at game startup: "./nexuiz-linux-whatever-binary +timedemo demos/demo1"

    Or you can create a benchmark config file (e.g. "benchmark.cfg" in the Nexuiz "data" directory):

    timedemo/demo1
    timedemo/demo1
    timedemo/demo1
    timedemo/demo2
    timedemo/demo2
    timedemo/demo2

    and do a "./nexuiz-linux-bla-bla +exec benchmark.cfg" to automate testing.

    Each run will print to console (ingame and to stdout) statistics like min/avg/max framerates.

    We'll release a new version of Nexuiz hopefully later this month (see http://emptyset.endoftheinternet.org/~polzer/nexuiz/builds/ for SVN builds) which will introduce (optional) rendering features like reflective water with refraction - this will make sure even fast cards are kept busy ;-)

    (Plus we finally have a much nicer menu ;-) )
    SavageX: Is there a command for Nexuiz similar to timedemoquit on the Quake games? Where when benchmarking is completed and results written to stdout that the game will immediately quit? I'm writing a Nexuiz 2.4 profile for the initial PTS release right now.

    Leave a comment:

Working...
X