Announcement

Collapse
No announcement yet.

How to time limit / strict loop limit runs?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to time limit / strict loop limit runs?

    Hi there,

    I am trying to benchmark a new software definied storage solution. So I have setup multiple vms on the compute cluster and I am able to run the same benchmark command to all the systems at the same time. I also set the environment variables for the TEST_RESULTS_IDENTIFIER, FORCE_TIMES_TO_RUN=12 but if I run for example phoronix-test-suite benchmark pts/aio-stress on all the vms simultaniously some still to 15 loops some even 40. which of course shifts the results since the load is less if one vm is already finished. Is there a workaround? For fio its even more important that all run for example 5mins rand read 4k then 5min rand write 4k etc. etc.

    Thanks for your AMAZING benchmarking tool!

    Regards,
    Yves

  • #2
    FORCE_TIMES_TO_RUN should run all that X amount of times with no less, no more. Are you saying you are seeing different behavior? If so, what PTS version are you on?
    Michael Larabel
    http://www.michaellarabel.com/

    Comment


    • #3
      Unfortunatly I am... I have set it to 10 which I can also see in the run it is writing
      Code:
      estimated trial run count: 10
      but if I do a benchmark the sample are very variable sometimes 10 sometimes 12 or even a lot more. But the runs are always MORE and the additional runs are always with a * in the name for example
      Code:
      Running Interim Test Script @ 11:47:20 Started Run 4 @ 11:47:22 *

      Version is Phoronix Test Suite v9.0.1

      Comment


      • #4
        This is from right now:

        Code:
        Flexible IO Tester 3.16:
            pts/fio-1.12.0 [Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory]
            Test 1 of 1
            Estimated Trial Run Count:    10           
            Estimated Time To Completion: 6 Minutes [11:50 CEST]
                Running Pre-Test Script @ 11:45:31
                Started Run 1 @ 11:45:31
                Running Interim Test Script @ 11:46:05
                Started Run 2 @ 11:46:07
                Running Interim Test Script @ 11:46:40
                Started Run 3 @ 11:46:42
                Running Interim Test Script @ 11:47:15
                Started Run 4 @ 11:47:17
                Running Interim Test Script @ 11:47:50
                Started Run 5 @ 11:47:52
                Running Interim Test Script @ 11:48:25
                Started Run 6 @ 11:48:27
                Running Interim Test Script @ 11:49:00
                Started Run 7 @ 11:49:02
                Running Interim Test Script @ 11:49:35
                Started Run 8 @ 11:49:37
                Running Interim Test Script @ 11:50:10
                Started Run 9 @ 11:50:12
                Running Interim Test Script @ 11:50:45
                Started Run 10 @ 11:50:47
                Running Interim Test Script @ 11:51:19
                Started Run 11 @ 11:51:21 *
                Running Interim Test Script @ 11:51:54
                Started Run 12 @ 11:51:56 *
                Running Interim Test Script @ 11:52:29
                Started Run 13 @ 11:52:31 *

        Comment


        • #5
          Hmmm, can you post the full command you are running? FORCE_TIMES_TO_RUN should never change, but guessing then that some other variable you might be setting could be overriding it.
          Michael Larabel
          http://www.michaellarabel.com/

          Comment


          • #6
            I don't think I have more variable set than FORCE_TIMES_TO_RUN and TEST_RESULTS_IDENTIFIER. I just run phoronix-test-suite benchmark pts/fio (rand read -> libaio -> no -> yes -> 4k -> default test path -> yes)

            Comment


            • #7
              Did you make any changes to your ~/.phoronix-test-suite/user-config.xml / (or if root, /etc/phoronix-test-suite.xml) files?
              Michael Larabel
              http://www.michaellarabel.com/

              Comment


              • #8
                we are now on 23 runs... ;-)

                Code:
                phoronix-test-suite benchmark pts/fio
                
                
                
                Phoronix Test Suite v9.0.1
                
                    Installed:     pts/fio-1.12.0
                
                
                Flexible IO Tester 3.16:
                    pts/fio-1.12.0
                    Disk Test Configuration
                        1: Random Read
                        2: Random Write
                        3: Sequential Read
                        4: Sequential Write
                        5: Test All Options
                        ** Multiple items can be selected, delimit by a comma. **
                        Type: 1
                
                
                        1: POSIX AIO
                        2: Sync
                        3: Linux AIO
                        4: Windows AIO
                        5: Test All Options
                        ** Multiple items can be selected, delimit by a comma. **
                        IO Engine: 3
                
                
                        1: Yes
                        2: No
                        3: Test All Options
                        ** Multiple items can be selected, delimit by a comma. **
                        Buffered: 2
                
                
                        1: No
                        2: Yes
                        3: Test All Options
                        ** Multiple items can be selected, delimit by a comma. **
                        Direct: 2
                
                
                        1:  4KB
                        2:  8KB
                        3:  16KB
                        4:  32KB
                        5:  64KB
                        6:  128KB
                        7:  256KB
                        8:  512KB
                        9:  1MB
                        10: 2MB
                        11: 4MB
                        12: 8MB
                        13: Test All Options
                        ** Multiple items can be selected, delimit by a comma. **
                        Block Size: 1
                
                
                
                System Information
                
                
                  PROCESSOR:          4 x Intel Xeon E312xx
                    Core Count:       4
                    Extensions:       SSE 4.2 + AVX
                    Cache Size:       4096 KB
                    Microcode:        0x1
                
                  GRAPHICS:           Red Hat QXL paravirtual graphic card
                    Screen:           1024x768
                
                  MOTHERBOARD:        oVirt Node
                    BIOS Version:     1.11.0-2.el7
                    Chipset:          Intel 440FX 82441FX PMC
                    Network:          Red Hat Virtio device
                
                  MEMORY:             4096MB
                
                  DISK:               107GB QEMU HDD + 11GB QEMU HDD
                    File-System:      ext4
                    Mount Options:    relatime rw seclabel
                    Disk Scheduler:   MQ-DEADLINE
                
                  OPERATING SYSTEM:   Fedora 30
                    Kernel:           5.0.9-301.fc30.x86_64 (x86_64)                                          
                    Desktop:          GNOME Shell 3.32.1                                                      
                    Display Server:   X Server                                                                
                    Compiler:         GCC 9.0.1 20190312                                                      
                    System Layer:     KVM                                                                     
                    Security:         SELinux                                                                 
                                      + l1tf: Mitigation of PTE Inversion                                     
                                      + meltdown: Mitigation of PTI                                           
                                      + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp   
                                      + spectre_v1: Mitigation of __user pointer sanitization                 
                                      + spectre_v2: Mitigation of Full generic retpoline IBPB: conditional IBRS_FW STIBP: disabled RSB filling
                
                    Would you like to save these test results (Y/n): rand_read_4k_test
                    Would you like to save these test results (Y/n): y
                
                    Recently Saved Test Results:
                       y                    [Today]
                       aio-simultan         [Today]
                       simultan-full-test   [Today]
                       simulatv4            [Today]
                       simultav3            [Today]
                
                    Enter a name for the result file: rand_read_4k_test
                
                Test Identifier: fedora1
                
                If desired, enter a new description below to better describe this result set / system configuration under test.
                Press ENTER to proceed without changes.
                
                Current Description: KVM testing on Fedora 30 via the Phoronix Test Suite.
                
                New Description: rand_read_4k_test
                
                Flexible IO Tester 3.16:
                    pts/fio-1.12.0 [Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory]
                    Test 1 of 1
                    Estimated Trial Run Count:    10           
                    Estimated Time To Completion: 5 Minutes [11:50 CEST]
                        Running Pre-Test Script @ 11:45:31
                        Started Run 1 @ 11:45:31
                        Running Interim Test Script @ 11:46:05
                        Started Run 2 @ 11:46:07
                        Running Interim Test Script @ 11:46:40
                        Started Run 3 @ 11:46:42
                        Running Interim Test Script @ 11:47:15
                        Started Run 4 @ 11:47:17
                        Running Interim Test Script @ 11:47:50
                        Started Run 5 @ 11:47:52
                        Running Interim Test Script @ 11:48:25
                        Started Run 6 @ 11:48:27
                        Running Interim Test Script @ 11:49:00
                        Started Run 7 @ 11:49:02
                        Running Interim Test Script @ 11:49:35
                        Started Run 8 @ 11:49:37
                        Running Interim Test Script @ 11:50:10
                        Started Run 9 @ 11:50:12
                        Running Interim Test Script @ 11:50:45
                        Started Run 10 @ 11:50:47
                        Running Interim Test Script @ 11:51:20
                        Started Run 11 @ 11:51:22 *
                        Running Interim Test Script @ 11:51:55
                        Started Run 12 @ 11:51:57 *
                        Running Interim Test Script @ 11:52:30
                        Started Run 13 @ 11:52:32 *
                        Running Interim Test Script @ 11:53:05
                        Started Run 14 @ 11:53:07 *
                        Running Interim Test Script @ 11:53:40
                        Started Run 15 @ 11:53:42 *
                        Running Interim Test Script @ 11:54:15
                        Started Run 16 @ 11:54:17 *
                        Running Interim Test Script @ 11:54:50
                        Started Run 17 @ 11:54:52 *
                        Running Interim Test Script @ 11:55:25
                        Started Run 18 @ 11:55:27 *
                        Running Interim Test Script @ 11:56:00
                        Started Run 19 @ 11:56:02 *
                        Running Interim Test Script @ 11:56:35
                        Started Run 20 @ 11:56:37 *
                        Running Interim Test Script @ 11:57:11
                        Started Run 21 @ 11:57:13 *
                        Running Interim Test Script @ 11:57:46
                        Started Run 22 @ 11:57:48 *
                        Running Interim Test Script @ 11:58:21
                        Started Run 23 @ 11:58:23 *

                Comment


                • #9
                  Originally posted by Michael View Post
                  Did you make any changes to your ~/.phoronix-test-suite/user-config.xml / (or if root, /etc/phoronix-test-suite.xml) files?
                  nope. nothing changed only ran these two commands
                  Code:
                  export TEST_RESULTS_IDENTIFIER=centos_vmware
                  export FORCE_TIMES_TO_RUN=10

                  Comment


                  • #10
                    Could you try a run without using TEST_RESULTS_IDENTIFIER to see if it behaves fine then? Otherwise will dive into the code this afternoon, but I use FORCE_TIMES_TO_RUN and have never hit that behavior, though I don't normally pair it with TEST_RESULTS_IDENTIFIER so guessing if that is causing some interference.
                    Michael Larabel
                    http://www.michaellarabel.com/

                    Comment

                    Working...
                    X