Metro redux tests failed to properly run

[leinardi]

Hi, I'm trying to run the metro redux tests but they fail without giving explanation.

I'm using the lastest git source:

$ git clone https://github.com/phoronix-test-suite/phoronix-test-suite.git

$ ./phoronix-test-suite benchmark 1501131-DE-METROREDU86

[PROBLEM] You must enter at least one test, suite, or result identifier to run. 

$ ./phoronix-test-suite benchmark metroll-redux metro2033-redux

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0
    Installed: pts/metro2033-redux-1.0.0

System Information

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440

    Would you like to save these test results (Y/n): n


Metro 2033 Redux 1.0:
    pts/metro2033-redux-1.0.0
    Test 1 of 2
    Estimated Trial Run Count:    3
    Estimated Test Run-Time:      5 Minutes
    Estimated Time To Completion: 14 Minutes
        Started Run 1 @ 21:51:46
        The test run did not produce a result.
        Started Run 2 @ 21:51:48
        The test run did not produce a result.
        Started Run 3 @ 21:51:50
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


Metro: Last Light Redux 1.0:
    pts/metroll-redux-1.0.0
    Test 2 of 2
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 10 Minutes
        Started Run 1 @ 21:51:57
        The test run did not produce a result.
        Started Run 2 @ 21:51:59
        The test run did not produce a result.
        Started Run 3 @ 21:52:01
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


The following tests failed to properly run:

	- pts/metro2033-redux-1.0.0: Phoronix Test Suite v5.6.0m1
	- pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1

Steam is running and the games run properly if started manually.

Any suggestion?

[Michael]

Try using the phoronix-test-suite debug-run <benchmark> command to see if any useful information is spewed.

[leinardi]

Thanks for the reply, this is the output:

$ ./phoronix-test-suite debug-run metroll-redux metro2033-redux

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0
    Installed: pts/metro2033-redux-1.0.0

========================================
Phoronix Test Suite v5.6.0m1
System Information
========================================

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440


========================================
Metro 2033 Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metro2033-redux-1.0.0/ && ./metro2033-redux  2>&1

./metro2033-redux: 2: cd: can't cd to /home/leinardi//.steam/steam/steamapps/common/Metro 2033 Redux
./metro2033-redux: 4: ./metro2033-redux: ./metro: not found

Test Result Value: 


The test run did not produce a result.


########################################
Metro 2033 Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################


========================================
Metro: Last Light Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metroll-redux-1.0.0/ && ./metroll-redux  2>&1

./metroll-redux: 2: cd: can't cd to /home/leinardi//.steam/steam/steamapps/common/Metro Last Light Redux
./metroll-redux: 4: ./metroll-redux: ./metro: not found

Test Result Value: 


The test run did not produce a result.


########################################
Metro: Last Light Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################



The following tests failed to properly run:

	- pts/metro2033-redux-1.0.0: Phoronix Test Suite v5.6.0m1
	- pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1

It seems to be a path problem due to differente case in directory names

PTS is looking for

.steam/steam/steamapps/common/Metro Last Light Redux

but my path has case:

.steam/steam/SteamApps/common/Metro\ Last\ Light\ Redux

[Michael]

Thanks for the reply, this is the output:

$ ./phoronix-test-suite debug-run metroll-redux metro2033-redux

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0
    Installed: pts/metro2033-redux-1.0.0

========================================
Phoronix Test Suite v5.6.0m1
System Information
========================================

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440


========================================
Metro 2033 Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metro2033-redux-1.0.0/ && ./metro2033-redux  2>&1

./metro2033-redux: 2: cd: can't cd to /home/leinardi//.steam/steam/steamapps/common/Metro 2033 Redux
./metro2033-redux: 4: ./metro2033-redux: ./metro: not found

Test Result Value: 


The test run did not produce a result.


########################################
Metro 2033 Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################


========================================
Metro: Last Light Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metroll-redux-1.0.0/ && ./metroll-redux  2>&1

./metroll-redux: 2: cd: can't cd to /home/leinardi//.steam/steam/steamapps/common/Metro Last Light Redux
./metroll-redux: 4: ./metroll-redux: ./metro: not found

Test Result Value: 


The test run did not produce a result.


########################################
Metro: Last Light Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################



The following tests failed to properly run:

    - pts/metro2033-redux-1.0.0: Phoronix Test Suite v5.6.0m1
    - pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1

It seems to be a path problem due to differente case in directory names

PTS is looking for

.steam/steam/steamapps/common/Metro Last Light Redux

but my path has case:

.steam/steam/SteamApps/common/Metro\ Last\ Light\ Redux

so looks like it comes down to steamapps vs. SteamApps..... Steam made that change within past few months to the Steam client... Have you been using Steam on that system for a while? I would have thought a Steam update would rename it for existing users. However, you can just modify ~/.phoronix-test-suite/test-profiles/pts/metro* to point to your Steam dir

[leinardi]

so looks like it comes down to steamapps vs. SteamApps..... Steam made that change within past few months to the Steam client... Have you been using Steam on that system for a while?

Since the closed beta (using the console trick )

When I get home I'll try to rename the directory according to the current case.

Thanks for the help

[Michael]

Since the closed beta (using the console trick )

When I get home I'll try to rename the directory according to the current case.

Thanks for the help

It was just in the past few months on new Steam installations that the change happened for me.... But yeah this is just one of the many examples why I don't like benchmarking in Steam due to losing the controlled environment.

[leinardi]

Hi,
after fixing the case the benchmarks run but no fps data is collected:

$ MONITOR=gpu.temp ./phoronix-test-suite benchmark metroll-redux metro2033-redux

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0
    Installed: pts/metro2033-redux-1.0.0

System Information

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440

    Would you like to save these test results (Y/n): 

Recently Saved Test Results:
- 1501131-DE-METROREDU86   [1 day old]
- 1412314-KH-LIGTI904750   [14 days old]

    Enter a name to save these results under: Asus GeForce GTX 980 4GB STRIX
    Enter a unique name to describe this test run / configuration: 

If you wish, enter a new description below to better describe this result set / system configuration under test.
Press ENTER to proceed without changes.

Current Description: Intel Core i7-4770K testing with a ASUS Z87-DELUXE and ASUS NVIDIA GeForce GTX 980 4096MB on Ubuntu 14.10 via the Phoronix Test Suite.

New Description: 

Sensors To Be Logged:
   - GPU Temperature


Metro 2033 Redux 1.0:
    pts/metro2033-redux-1.0.0
    Test 1 of 2
    Estimated Trial Run Count:    3
    Estimated Test Run-Time:      5 Minutes
    Estimated Time To Completion: 14 Minutes
        Started Run 1 @ 17:52:11
        The test run did not produce a result.
        Started Run 2 @ 17:53:45
        The test run did not produce a result.
        Started Run 3 @ 17:55:01
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


Metro: Last Light Redux 1.0:
    pts/metroll-redux-1.0.0
    Test 2 of 2
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 10 Minutes
        Started Run 1 @ 17:56:51
        The test run did not produce a result.
        Started Run 2 @ 18:00:06
        The test run did not produce a result.
        Started Run 3 @ 18:03:04
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


The following tests failed to properly run:

	- pts/metro2033-redux-1.0.0: Phoronix Test Suite v5.6.0m1
	- pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1



Finishing System Sensor Monitoring Process
    Do you want to view the results in your web browser (Y/n): 
    Would you like to upload the results to OpenBenchmarking.org (Y/n): 

Results Uploaded To: http://openbenchmarking.org/result/1501157-DE-ASUSGEFOR65
    Do you want to launch OpenBenchmarking.org (Y/n):

$ ./phoronix-test-suite debug-run metroll-redux 

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0

========================================
Phoronix Test Suite v5.6.0m1
System Information
========================================

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440


========================================
Metro: Last Light Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metroll-redux-1.0.0/ && ./metroll-redux  2>&1

ERROR: ld.so: object '/home/leinardi/.phoronix-test-suite/installed-tests/pts/libframetime-1.0.0//libframetime/libframetime.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Setting breakpad minidump AppID = 287390
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561197970610211 [API loaded no]

Test Result Value: 


The test run did not produce a result.


########################################
Metro: Last Light Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################



The following tests failed to properly run:

	- pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1

There is no libframetime.so in /home/leinardi/.phoronix-test-suite/installed-tests/pts/libframetime-1.0.0/libframetime/.

[Michael]

Hi,
after fixing the case the benchmarks run but no fps data is collected:

$ MONITOR=gpu.temp ./phoronix-test-suite benchmark metroll-redux metro2033-redux

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0
    Installed: pts/metro2033-redux-1.0.0

System Information

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440

    Would you like to save these test results (Y/n): 

Recently Saved Test Results:
- 1501131-DE-METROREDU86   [1 day old]
- 1412314-KH-LIGTI904750   [14 days old]

    Enter a name to save these results under: Asus GeForce GTX 980 4GB STRIX
    Enter a unique name to describe this test run / configuration: 

If you wish, enter a new description below to better describe this result set / system configuration under test.
Press ENTER to proceed without changes.

Current Description: Intel Core i7-4770K testing with a ASUS Z87-DELUXE and ASUS NVIDIA GeForce GTX 980 4096MB on Ubuntu 14.10 via the Phoronix Test Suite.

New Description: 

Sensors To Be Logged:
   - GPU Temperature


Metro 2033 Redux 1.0:
    pts/metro2033-redux-1.0.0
    Test 1 of 2
    Estimated Trial Run Count:    3
    Estimated Test Run-Time:      5 Minutes
    Estimated Time To Completion: 14 Minutes
        Started Run 1 @ 17:52:11
        The test run did not produce a result.
        Started Run 2 @ 17:53:45
        The test run did not produce a result.
        Started Run 3 @ 17:55:01
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


Metro: Last Light Redux 1.0:
    pts/metroll-redux-1.0.0
    Test 2 of 2
    Estimated Trial Run Count:    3
    Estimated Time To Completion: 10 Minutes
        Started Run 1 @ 17:56:51
        The test run did not produce a result.
        Started Run 2 @ 18:00:06
        The test run did not produce a result.
        Started Run 3 @ 18:03:04
        The test run did not produce a result.

    Test Results:

    Average: 0 Frames Per Second
    This test failed to run properly.


The following tests failed to properly run:

    - pts/metro2033-redux-1.0.0: Phoronix Test Suite v5.6.0m1
    - pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1



Finishing System Sensor Monitoring Process
    Do you want to view the results in your web browser (Y/n): 
    Would you like to upload the results to OpenBenchmarking.org (Y/n): 

Results Uploaded To: http://openbenchmarking.org/result/1501157-DE-ASUSGEFOR65
    Do you want to launch OpenBenchmarking.org (Y/n):

$ ./phoronix-test-suite debug-run metroll-redux 

Phoronix Test Suite v5.6.0m1

    Installed: pts/libframetime-1.0.0
    Installed: pts/metroll-redux-1.0.0

========================================
Phoronix Test Suite v5.6.0m1
System Information
========================================

Hardware:
Processor: Intel Core i7-4770K @ 3.90GHz (8 Cores), Motherboard: ASUS Z87-DELUXE, Chipset: Intel 4th Gen Core DRAM, Memory: 16384MB, Disk: 1000GB Samsung SSD 840, Graphics: ASUS NVIDIA GeForce GTX 980 4096MB (1177/3505MHz), Audio: Intel 8 /C220, Monitor: LG LG TV, Network: Intel Connection I217-V + Qualcomm Atheros AR93xx Wireless

Software:
OS: Ubuntu 14.10, Kernel: 3.16.0-29-generic (x86_64), Desktop: Unity 7.3.1, Display Server: X Server 1.16.1.901 (1.16.2 RC 1), Display Driver: NVIDIA 343.36, OpenGL: 4.4.0, Compiler: GCC 4.9.1, File-System: ext4, Screen Resolution: 4480x1440


========================================
Metro: Last Light Redux (Run 1 of 1)
========================================


Test Run Command: cd /home/leinardi/.phoronix-test-suite/installed-tests/pts/metroll-redux-1.0.0/ && ./metroll-redux  2>&1

ERROR: ld.so: object '/home/leinardi/.phoronix-test-suite/installed-tests/pts/libframetime-1.0.0//libframetime/libframetime.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Setting breakpad minidump AppID = 287390
Steam_SetMinidumpSteamID:  Caching Steam ID:  76561197970610211 [API loaded no]

Test Result Value: 


The test run did not produce a result.


########################################
Metro: Last Light Redux:
Phoronix Test Suite v5.6.0m1


Average: 0 Frames Per Second
########################################



The following tests failed to properly run:

    - pts/metroll-redux-1.0.0: Phoronix Test Suite v5.6.0m1

There is no libframetime.so in /home/leinardi/.phoronix-test-suite/installed-tests/pts/libframetime-1.0.0/libframetime/.

I guess try building libframetime yourself and see what's wrong.... Should just be a matter of running make inside the directory.

[leinardi]

I guess try building libframetime yourself and see what's wrong....

Yep, I was missing EGL/egl.h.

sudo apt-get install libegl1-mesa-dev solved the problem.

I finally managed to properly finish a benchmark but the performance of my system sucks

http://openbenchmarking.org/result/1501160-DE-STRIXGTX976

Only 61.78 FPS on Metro:LL??

My game settings are 1920x1080, very high quality, AA 2x, AF 16x, normal motion blur, VSync off, advanced PhysX on, vertical fov 59.

I'm using NVIDIA 343.36 instead of the latest beta because with the 346.22 on very high I got some strange glitch.

When I run the benchmark what settings are used? The script defines them or use those of my system?

Didn't you have any glitches using 346.22 and very high settings?

[leinardi]

When I run the benchmark what settings are used? The script defines them or use those of my system?

Ok, I answer myself this question: the benchmark runs with the user settings.

I lowered the settings to 1920x1080, low quality, no AA, AF 4x, no motion blur, no VSync, no advanced PhysX and hit an average of 142.82 fps on Metro:LL:

http://openbenchmarking.org/result/1501160-DE-STRIXGTX969

I fear that the benchmark of Metro Redux are not useful for comparison if you do not find a way to always set the same settings for all.