Announcement

Collapse
No announcement yet.

Broken tests in PTS 3.0.0

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Broken tests in PTS 3.0.0

    I am attempting to use PTS 3.0.0 to benchmark a new server I got at work. This is a Dell R910 with 32 cores (64 with SMT which I have disabled for these tests), and 256GB of DDR3. I am running the universe-cli suite, as I do not intend to install a GUI on the system. I have run into several tests failing to run, and thought it might be good to report the errors here so that they may be fixed in a future version of PTS.

    First off, two tests which fail with >8CPUs, mencoder & ffmpeg. Neither of these tests like it when you tell them to use more than 8 threads. As I mentioned above, the system I'm testing happens to have 32 cores, so using $NUM_CPU_CORES causes the tests to fail.

    Next, compress-pbzip2 which as far as I can tell seems to work when I run the command line by hand, but tells me "The test did not produce a result" each time it runs. I think this is a timing issue (as in, the test isn't timed correctly, or assumes that the result will be greater than specific value and is throwing away the results because they're too fast?).

    I also had an issue with a couple of Compile Bench tests, Compile and Read Compiled Tree, though the Initial Create test worked fine. I think this is a parsing error, as the results I get from running the test in debug mode seem to indicate that both the compile & the read tree worked correctly:
    Code:
    run complete:
    ==========================================================================
    intial create total runs 10 avg 146.98 MB/s (user 0.59s sys 0.91s)
    no runs for create
    no runs for patch
    compile total runs 10 avg 792.47 MB/s (user 0.12s sys 0.74s)
    no runs for clean
    no runs for read tree
    read compiled tree total runs 3 avg 826.78 MB/s (user 0.38s sys 0.71s)
    no runs for delete tree
    delete compiled tree total runs 10 avg 0.95 seconds (user 0.25s sys 0.69s)
    no runs for stat tree
    no runs for stat compiled tree
    Next is mrbayes, which is another case of too many CPUs:
    Code:
          The number of chains must be at least as great
          as the number of processors (32)
    Finally, java-scimark2 FFT test failed as well. This appears to be another parsing error, as the test definitely produces results:
    Code:
    ========================================
    Java SciMark (Run 4 of 4)
    ========================================
    
    
    Test Run Command: cd /home/thalin/.phoronix-test-suite/installed-tests/pts/java-scimark2-1.1.1/ && ./java-scimark2 TEST_FFT 2>&1
    
    
    SciMark 2.0a
    
    Composite Score: 865.7334474774465
    FFT (1024): 538.1797821272396
    SOR (100x100):   963.7184161416388
    Monte Carlo : 459.6497601801464
    Sparse matmult (N=1000, nz=5000): 867.1650672662963
    LU (100x100): 1499.954211671911
    
    java.vendor: Sun Microsystems Inc.
    java.version: 1.6.0_20
    os.arch: amd64
    os.name: Linux
    os.version: 2.6.35-25-server
    
    Result Parsing Search Key: 
    FFT (1024): 
    
    
    Result Line: Composite Score: 865.7334474774465
    
    
    Test Result Value: 
    
    
    The test did not produce a result.
    I hope that these reports are useful, and if any further information is required I will be happy to provide it.

  • #2
    Sounds like that would be a fun system to benchmark ...Too many CPUs...

    Comment


    • #3
      Sry for using this thread as a dump for other bugs

      There also seems to be a bug in the Bullet Physics benchmark. Using Fedora 14 the last test always fails, cause a new line is added using
      Code:
      echo "\n" >> $LOG_FILE
      . But to make echo interpret this as a new line, one has to use
      Code:
      echo -e "\n" >> $LOG_FILE
      instead.

      By the way, for compiling bullet with clang, one has to patch the source (http://http://code.google.com/p/bull.../detail?r=2222). Perhaps one could include this patch in the install.sh.

      Comment


      • #4
        Another bug.

        Nero2d fails to compile on Fedora 14 cause there are some hardcoded symlinks in the doc part of the package.

        i modified the corresponding install.sh to patch the makefile the way that the docs arent build

        Code:
        #!/bin/sh
        
        mkdir $HOME/nero2d_
        
        tar -zxvf nero2d-2.0.2.tar.gz
        
        patch -p0 <<'EOT'
        --- nero2d-2.0.2.orig/src/nexus/nexus.cpp	2009-04-03 09:42:29.000000000 -0400
        +++ nero2d-2.0.2/src/nexus/nexus.cpp	2009-06-13 21:52:42.589304348 -0400
        @@ -21,6 +21,8 @@
         #include "nexus.h"
         #include "engine.h"
         #include "base.h"
        +#include <cstring>
        +#include <cstdio>
         
         using namespace std;
         
        EOT
        
        
        patch -p0 <<'EOT'
        --- nero2d-2.0.2.orig/Makefile.in    2009-04-03 15:42:34.000000000 +0200
        +++ nero2d-2.0.2/Makefile.in  2011-03-02 19:23:12.179899327 +0100
        @@ -193,7 +193,7 @@
         # not a GNU package. You can remove this line, if
         # have all needed files, that a GNU package needs
         AUTOMAKE_OPTIONS = foreign 1.4
        -SUBDIRS = config benchmark doc examples src
        +SUBDIRS = config benchmark examples src
         all: config.h
                $(MAKE) $(AM_MAKEFLAGS) all-recursive
        
        EOT
        
        cd nero2d-2.0.2/
        ./configure --prefix=$HOME/nero2d_
        make -j $NUM_CPU_JOBS
        echo $? > ~/install-exit-status
        make install
        cd ..
        rm -rf nero2d-2.0.2/
        
        echo "#!/bin/sh
        ./nero2d_/bin/nero2d \$@ > \$LOG_FILE 2>&1
        echo \$? > ~/test-exit-status" > nero2d
        chmod +x nero2d

        Comment

        Working...
        X