Announcement

Collapse
No announcement yet.

pts/server brings many failed tests, but why?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • pts/server brings many failed tests, but why?

    Hi,

    I want to test different hardware servers under different hypervisors (VMware, XCP-ng, Proxmox and so on).

    On my fresh installed Ubuntu 20.04 and Phoronix 9.6.0 as root with all dependencies installed the pts/server suite brings many failed tests at the end. I still do not understand why.

    Code:
    The following tests failed to properly run:
    
    - pts/blogbench-1.1.0
    - pts/apache-siege-1.0.4: Concurrent Users: 200
    - pts/apache-siege-1.0.4: Concurrent Users: 250
    - pts/mysqlslap-1.1.0: Clients: 64
    - pts/mysqlslap-1.1.0: Clients: 256
    - pts/pgbench-1.9.1: Scaling: Buffer Test - Test: Normal Load - Mode: Read Write
    - pts/pgbench-1.9.1: Scaling: Buffer Test - Test: Normal Load - Mode: Read Only
    - pts/pgbench-1.9.1: Scaling: Buffer Test - Test: Heavy Contention - Mode: Read Write
    - pts/pgbench-1.9.1: Scaling: Buffer Test - Test: Heavy Contention - Mode: Read Only
    - pts/mcperf-1.3.0: Method: Get - Connections: 1
    - pts/mcperf-1.3.0: Method: Get - Connections: 4
    - pts/mcperf-1.3.0: Method: Get - Connections: 16
    - pts/mcperf-1.3.0: Method: Get - Connections: 32
    - pts/mcperf-1.3.0: Method: Get - Connections: 64
    - pts/mcperf-1.3.0: Method: Get - Connections: 128
    - pts/mcperf-1.3.0: Method: Get - Connections: 256
    - pts/mcperf-1.3.0: Method: Set - Connections: 1
    - pts/mcperf-1.3.0: Method: Set - Connections: 4
    - pts/mcperf-1.3.0: Method: Set - Connections: 16
    - pts/mcperf-1.3.0: Method: Set - Connections: 32
    - pts/mcperf-1.3.0: Method: Set - Connections: 64
    - pts/mcperf-1.3.0: Method: Set - Connections: 128
    - pts/mcperf-1.3.0: Method: Set - Connections: 256
    - pts/mcperf-1.3.0: Method: Delete - Connections: 1
    - pts/mcperf-1.3.0: Method: Delete - Connections: 4
    - pts/mcperf-1.3.0: Method: Delete - Connections: 16
    - pts/mcperf-1.3.0: Method: Delete - Connections: 32
    - pts/mcperf-1.3.0: Method: Delete - Connections: 64
    - pts/mcperf-1.3.0: Method: Delete - Connections: 128
    - pts/mcperf-1.3.0: Method: Delete - Connections: 256
    - pts/mcperf-1.3.0: Method: Add - Connections: 1
    - pts/mcperf-1.3.0: Method: Add - Connections: 4
    - pts/mcperf-1.3.0: Method: Add - Connections: 16
    - pts/mcperf-1.3.0: Method: Add - Connections: 32
    - pts/mcperf-1.3.0: Method: Add - Connections: 64
    - pts/mcperf-1.3.0: Method: Add - Connections: 128
    - pts/mcperf-1.3.0: Method: Add - Connections: 256
    - pts/mcperf-1.3.0: Method: Replace - Connections: 1
    - pts/mcperf-1.3.0: Method: Replace - Connections: 4
    - pts/mcperf-1.3.0: Method: Replace - Connections: 16
    - pts/mcperf-1.3.0: Method: Replace - Connections: 32
    - pts/mcperf-1.3.0: Method: Replace - Connections: 64
    - pts/mcperf-1.3.0: Method: Replace - Connections: 128
    - pts/mcperf-1.3.0: Method: Replace - Connections: 256
    - pts/mcperf-1.3.0: Method: Append - Connections: 1
    - pts/mcperf-1.3.0: Method: Append - Connections: 4
    - pts/mcperf-1.3.0: Method: Append - Connections: 16
    - pts/mcperf-1.3.0: Method: Append - Connections: 32
    - pts/mcperf-1.3.0: Method: Append - Connections: 64
    - pts/mcperf-1.3.0: Method: Append - Connections: 128
    - pts/mcperf-1.3.0: Method: Append - Connections: 256
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 1
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 4
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 16
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 32
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 64
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 128
    - pts/mcperf-1.3.0: Method: Prepend - Connections: 256
    - pts/cassandra-1.0.3: Test: Reads
    - pts/cassandra-1.0.3: Test: Mixed 1:1
    - pts/cassandra-1.0.3: Test: Mixed 1:3
    - pts/hbase-1.0.1: Test: Random Write - Clients: 1
    - pts/hbase-1.0.1: Test: Random Write - Clients: 4
    - pts/hbase-1.0.1: Test: Random Write - Clients: 16
    - pts/hbase-1.0.1: Test: Random Write - Clients: 32
    - pts/hbase-1.0.1: Test: Random Write - Clients: 64
    - pts/hbase-1.0.1: Test: Random Write - Clients: 128
    - pts/hbase-1.0.1: Test: Random Write - Clients: 256
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 1
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 4
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 16
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 32
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 64
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 128
    - pts/hbase-1.0.1: Test: Async Random Write - Clients: 256
    - pts/hbase-1.0.1: Test: Random Read - Clients: 1
    - pts/hbase-1.0.1: Test: Random Read - Clients: 4
    - pts/hbase-1.0.1: Test: Random Read - Clients: 16
    - pts/hbase-1.0.1: Test: Random Read - Clients: 32
    - pts/hbase-1.0.1: Test: Random Read - Clients: 64
    - pts/hbase-1.0.1: Test: Random Read - Clients: 128
    - pts/hbase-1.0.1: Test: Random Read - Clients: 256
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 1
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 4
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 16
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 32
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 64
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 128
    - pts/hbase-1.0.1: Test: Async Random Read - Clients: 256
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 1
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 4
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 16
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 32
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 64
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 128
    - pts/hbase-1.0.1: Test: Sequential Write - Clients: 256
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 1
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 4
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 16
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 32
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 64
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 128
    - pts/hbase-1.0.1: Test: Sequential Read - Clients: 256
    - pts/hbase-1.0.1: Test: Increment - Clients: 1
    - pts/hbase-1.0.1: Test: Increment - Clients: 4
    - pts/hbase-1.0.1: Test: Increment - Clients: 16
    - pts/hbase-1.0.1: Test: Increment - Clients: 32
    - pts/hbase-1.0.1: Test: Increment - Clients: 64
    - pts/hbase-1.0.1: Test: Increment - Clients: 128
    - pts/hbase-1.0.1: Test: Increment - Clients: 256
    Detailed debug-benchmark example for pts/server (for the blogbench test run)

    Code:
    ========================================
    BlogBench (Run 1 of 1)
    ========================================
    
    
    Test Run Command: cd /var/lib/phoronix-test-suite/installed-tests/pts/blogbench-1.1.0/ && ./blogbench 2>&1
    
    
    Frequency = 10 secs
    Scratch dir = [/var/lib/phoronix-test-suite/installed-tests/pts/blogbench-1.1.0//scratch]
    Spawning 3 writers...
    Spawning 1 rewriters...
    Spawning 5 commenters...
    Spawning 100 readers...
    Benchmarking for 30 iterations.
    The test will run during 5 minutes.
    
    Nb blogs R articles W articles R pictures W pictures R comments W comments
    765 2109482 39563 1437440 38788 903026 72176
    1148 2162055 21919 1477712 20942 984356 43925
    1512 2170516 21159 1487922 19839 1008404 42751
    1877 2226106 20899 1529475 19853 1061128 42545
    2224 2154031 20598 1496055 19788 994654 35185
    2582 2091998 21829 1448854 19210 976793 40336
    2955 2144429 21517 1484065 19644 988148 37027
    3330 2146309 22660 1496238 21006 992295 33169
    3683 2097889 20887 1457541 20149 948628 36808
    4049 2141985 21076 1497831 19696 959197 36795
    4419 2076129 21376 1459892 20898 953305 41333
    4762 2145670 20573 1505589 18778 979357 35926
    5074 4229033 19803 1464216 17469 964373 37089
    5348 1766341 16842 1244530 16389 812179 33149
    5631 1470204 16403 1033168 15385 688086 32955
    5799 978613 11487 690249 10459 472871 41676
    5920 862194 8894 607487 7616 419768 38092
    6021 760746 8249 534361 6474 388973 33196
    6127 694702 7773 487914 6744 357874 33512
    6228 657251 8122 463625 6399 344213 31464
    6313 588556 6707 413481 5869 310963 29057
    6396 566554 6543 397936 5113 308895 27252
    6484 553720 7090 387925 5332 298897 30501
    6556 513290 5602 359542 4527 283012 29249
    6629 494587 5818 348222 5044 275289 26033
    6706 476339 6004 336231 4762 277005 25950
    6777 415253 5598 294945 4877 242901 22606
    6845 437169 5408 310719 4529 252197 23289
    6924 444836 6009 316203 4901 253103 25789
    6995 393655 5853 279927 4537 229770 22105
    
    Final score for writes: 6995
    Final score for reads : 795953
    
    
    Failed Initial Check For Matching: not in READ
    
    
    Failed Initial Check For Matching: not in WRITE
    
    
    The test run did not produce a result.
    
    
    Log File At: /var/lib/phoronix-test-suite/installed-tests/pts/blogbench-1.1.0/blogbench-1.1.0-1590262609-1.log
    The "result lines" are correct and matches with "/var/lib/phoronix-test-suite/test-profiles/pts/blogbench-1.1.0/results-definition.xml" I cannot understand "Failed Initial Check For Matching".

    Code:
    phoronix-test-suite debug-benchmark pts/blogbench-1.1.0
    brings this output, slightly different (errors also here, but still results)

    Code:
    # phoronix-test-suite debug-benchmark pts/blogbench-1.1.0
    
    
    Evaluating External Test Dependencies .................................................. .................................................. .................................................. .........................................
    
    Phoronix Test Suite v9.6.0
    
    Installed: pts/blogbench-1.1.0
    
    
    BlogBench 1.1:
    pts/blogbench-1.1.0
    Disk Test Configuration
    1: Read
    2: Write
    3: Test All Options
    ** Multiple items can be selected, delimit by a comma. **
    Test: 3
    
    
    ========================================
    Phoronix Test Suite v9.6.0
    System Information
    ========================================
    
    
    PROCESSOR: 8 x Intel Xeon E5-2620 v4
    Core Count: 8
    Extensions: SSE 4.2 + AVX2 + AVX + RDRAND + FSGSBASE
    Cache Size: 20480 KB
    Microcode: 0xb000038
    
    GRAPHICS: Cirrus Logic GD 5446
    Screen: 1024x768
    
    MOTHERBOARD: Xen HVM domU v4.13
    BIOS Version: 4.13
    Chipset: Intel 440FX 82441FX PMC
    
    MEMORY: 1 x 16380 MB RAM
    
    DISK: 49GB
    File-System: ext4
    Mount Options: relatime rw
    Disk Scheduler: MQ-DEADLINE
    
    OPERATING SYSTEM: Ubuntu 20.04
    Kernel: 5.4.0-31-generic (x86_64)
    Compiler: GCC 9.3.0
    System Layer: Xen HVM domU 4.13
    Security: itlb_multihit: KVM: Vulnerable
    + l1tf: Mitigation of PTE Inversion
    + mds: Mitigation of Clear buffers; SMT Host state unknown
    + meltdown: Mitigation of PTI
    + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp
    + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization
    + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling
    + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown
    
    
    ========================================
    BlogBench (Run 1 of 1)
    ========================================
    
    
    Test Run Command: cd /var/lib/phoronix-test-suite/installed-tests/pts/blogbench-1.1.0/ && ./blogbench READ 2>&1
    
    ########################################
    BlogBench:
    Test: Read
    
    777600 Final Score
    
    Average: 777600 Final Score
    ########################################
    
    
    Result compared to 799 OpenBenchmarking.org samples since 4 December 2018; median result: 514314. Box plot of samples:
    [---------##############*#######!#####*######*##*########### ####################----*----------------------*-*---------*----------------------------------| ]
    This Result (60th Percentile): 777600 ^
    ADATA SX8200PNP: 727149 ^ 8 x 300GB H106030SDSUN300G: 1360222 ^ 4 x 2048GB MR9460-16i: 1930331 ^
    ^ Samsung SSD 970 EVO Plus: 606079 Samsung SSD 970 PRO: 1768747 ^
    ^ INTEL SSDPEKNW512G8: 382654 INTEL SSDPED1D280GA: 1723568 ^
    
    Failed Initial Check For Matching: WRITE not in READ
    
    
    Result Key: #_RESULT_#
    
    
    Template Line: Final score for writes: #_RESULT_#
    
    
    Result Parsing Search Key: "writes"
    
    
    Result Line: Final score for writes: 7153
    
    
    Test Result Parser Returning: 7153
    
    
    ########################################
    BlogBench:
    Test: Write
    
    7153 Final Score
    
    Average: 7153 Final Score
    ########################################
    
    
    Result compared to 797 OpenBenchmarking.org samples since 4 December 2018; median result: 3631. Box plot of samples:
    [|-*---*---*-##!##############*###############################* #################*#####*####*#---------------------------------------------------*-------------------------*------*-*----------------------------------------| ]
    ^ This Result (57th Percentile): 7153
    ^ 2 x 32GB SATA SSD: 2575 SSDSCKJB120G7R: 14978 ^ Samsung SSD 970 EVO: 22087 ^ Samsung SSD 970 PRO: 34899 ^ INTEL SSDPED1D280GA: 43456 ^
    ^ INTEL SSDSC2KF25: 1793 TOSHIBA-RD400: 20845 ^ Force MP600: 42771 ^
    ^ QEMU HDD: 815 INTEL SSDPE2ME800G4: 19332 ^ INTEL SSDPED1D960GAY: 41153 ^
    Other tests such as mysql etc. seem to have their own problems such as the daemon not starting or its socketfile not being found under /tmp/..., again interestingly the test works when called directly and manually.

    Is there any documentation that can help you a bit more with debugging?
    Last edited by Edur; 05-23-2020, 04:15 PM.

  • #2
    The blogbench issue should be fixed next time you run `phoronix-test-suite openbenchmarking-refresh` and then run the server suite, it wasn't passing some needed arguments from the test suite definition.

    For other tests that work when called outside of the server suite, would need to know which ones you were able to reproduce that behavior for as opposed to not being able to run at all. All of these tests at least on my side run fine on Ubuntu 20.04. Note that some tests like mysqlslap may have issues running as root due to their default configuration in the name of security.
    Michael Larabel
    http://www.michaellarabel.com/

    Comment

    Working...
    X