Tweet
It's only been two weeks and one day since Phoronix Test Suite 1.2.0 was released, but with a lot of work already getting done for Orkdal in the git tree, it's time already for the first alpha build of Phoronix Test Suite 1.4.0. First though, it's important to note that the final release of Phoronix Test Suite 1.4 "Orkdal" will not be released until December or so of this year and there are still many additional changes in the works. Before going gold, expect at least three more development releases. We're certainly off to a great start though.
Below is the official change-log between Phoronix Test Suite 1.2 and 1.4 Alpha 1, which is about 50 changes. Here though are the most important ones:
- In addition to having tests and suites stored within their conventional locations (phoronix-test-suite/pts/ or /usr/share/phoronix-test-suite/), they can now be stored within ~/.phoronix-test-suite/test-profiles/ / ~/.phoronix-test-suite/test-resources/ and ~/.phoronix-test-suite/test-suites/ respectively. This makes it easy for a user to install a new or updated test without needing root access and it allows for the easy installation of third-party tests. These directories will also be used for automatically installing some updated tests where available.
- Cascading Test Profiles: This is a new way of writing test profiles that would be similar to the style of Cascading Style Sheets or inheritance in many programming languages. A test profile is now able to use an XML tag of "Extends"that makes it extend another test. For example, all of the WINE-based tests are going to extend a generic WINE test profile that all it does is build the WINE release and sets up the environment. Then all of the other WINE test profiles don't need to worry about this process.Cascading Test Profiles can also be used for sharing other common resources between tests. In Orkdal Alpha 1 there are three base tests currently being extended, for the timed audio encoding, timed build, and video tests. Due to Cascading Test Profiles, the pts-shared test resource has been dropped in this release.
- Self-Contained Test Profiles: This feature allows an entire test profile with test resources to be bundled into a single SCTP file. In doing so, it makes it very easy to distribute small tests. For instance, if you notice a bug with some program and are able to automate it with a Phoronix Test Suite profile, you can write a simple test and throw it into an SCTP file. Following that, just attach it to the BugZilla or what have you. Then any user with the Phoronix Test Suite installed just needs to run./test-name.sctp and it will automatically initialize the Phoronix Test Suite and begin running the test. Self-Contained Test Profiles makes testing very portable and is really intended for just small tests that in many cases would be temporary and are not intended for submission into the Phoronix Test Suite program itself. The Self-Contained Test Profiles feature is implemented, but with only limited testing thus far and needs some documentation on how to write a SCTP.
- The Phoronix Test Suite temp directory is no longer used when downloading files.
- A World of Padman test has been added (named: padman).
There's a lot of other changes in this release, but these are the main changes exposed to the end-user within Alpha 1.
Download: http://www.phoronix-test-suite.com/d...-suite-1.4.0a1
Phoronix Test Suite 1.4.0 Alpha 1
September 18, 2008
- pts-core: Change test/suite type strings to define statements
- pts-core: Add test type caching support
- pts-core: Create the unified PTS test/suite type handler
- pts-core: Clean-up some test installation functions
- pts-core: Provide new architecture for test/suite checking through the type handler
- pts-core: Drop PTS_USER_DIR usage
- pts-core: Add support for having tests and suites stored locally within ~/.phoronix-test-suite/ as an option for users installing new tests/suites or third-party files
- pts-core: Add support so that a test profile can extend another test, for Cascading Test Profile support. This is similar to inheritance in programming languages or Cascading Style Sheets.
- pts-core: Add in support for Self-Contained Test Profiles (SCTP) that initiate the Phoronix Test Suite and runs a stand-alone test
- pts-core: A test profile isn't real: likely incomplete or just a base test within a Self-Contained Test Profile extension
- pts-core: Report identifiers of test profiles being extended to the test profiles being run
- pts-core: Drop PTS shared download support for test profiles, instead they should be using the Cascading Test Profiles functionality
- pts-core: Don't allow the user to save a test result file if it's the same name as a real test profile or test suite
- pts-core: Add Mac OS X operating system detection support (using the IS_MACOSX define)
- pts-core: Improve VirtualBox virtualization detection for guest operating systems running
- pts-core: Add $COMPILER and $VIDEO_DRIVER user variables when saving the results and supplying the test identifier
- pts-core: Let the size of files being downloaded be specified individually within the test's downloads.xml profile instead of a generic one through the test profile XML file
- pts-core: When downloading files for tests, don't first download the file to the PTS temporary directory, but just rename it initially within the test's environment directory
- pts-core: When running a test (or suite that contains a test) on an unsupported architecture or platform, don't abruptly exit
- pts-core: Add module_setup() to pts_module_interface for returning an array of objects that can be configured by the end-user
- pts-core: Add support for persistent configuration options that end-users can easily adjust for PTS modules by running "phoronix-test-suite module-setup MODULE-NAME" that are stored to ~/.phoronix-test-suite/modules-config.xml
- pts-core: Add pts_module::set_option() function to allow a module itself to set a persistent configuration option
- pts-core: Add PTS_QUIT module return type for having the Phoronix Test Suite stop immediately and prematurely (this should only be used in the event of reaching a fatal bug or other problem)
- pts-core: Have the Phoronix Test Suite check once a day to see if the version in use is the latest version, otherwise inform the user of an available update
- pts-core: Add fall-back support for reading the PCI subsystem vendor if the motherboard vendor can't be determined using the other HAL information
- pts-core: Add a PreInstallMessage tag for test profiles that will be shown to the user prior to installing the test
- pts-core: Don't have PreRunMessage or PreInstallMessage hang when in batch mode
- pts-core: pts_find_file() -> pts_find_result_file()
- pts-core: Add initial support for detecting the motherboard Southbridge where applicable
- pts-core: Omit OEM in read_hal()
- pts-core: With list-tests, don't list tests that aren't compliant with the running system's architecture and platform
- pts-core: Add support so test options can be set as an environment variable by using <TEST NAME UPPER_CASE>_<NUM OPTION>=<INDEX OF REQUEST>
- pts-core: Switch to using php_uname() in pts-init.php
- pts-core: Add support for reading all ATI forced anti-aliasing types: Box, Narrow-tent, Wide-tent, and Edge-detect
- pts: Switch all audio encoding tests to extend a timed-audio-encode base test instead of all tests having some of the same code copied throughout
- pts: Switch all build tests to extend a timed-build base test instead of all tests having some of the same code copied throughout
- pts: Switch all video tests to extend a video-sample base test instead of all tests having some of the same code copied throughout
- pts: Change all current and relevant test profiles shipping the Phoronix Test Suite to using the FileSize tags within their downloads.xml files
- pts: Improve the reported environment size for many of the test profiles
- pts: Update compress-7zip test profile to use p7zip 4.58
- pts: Add World of Padman test profile (named padman)
- pts: Add padman to ioquake3-games test suite
- pts: Update Mandriva Linux packages XML file (Thanks To St?phane T?letch?a)
- tandem_Xml: Add a pts_test_tandem_XmlReader object that extends tandem_XmlReader but takes advantage of the handleXmlZeroTagFallback
- tandem_Xml: Add fallback function handler support when a tag isn't found
- graphics_event_checker: Add graphics_event_checker module that makes sure the graphics system isn't messed up or having unintended options enabled while tests are running
- graphics_event_checker: Merge the gpu_error_counter module functionality into the graphics_event_checker module
- timed_screenshot: A module that takes a screenshot while tests are running at a pre-defined interval
- documentation: Add basic documentation concerning the Self-Contained Test Profiles (SCTP) XML tags that are added
Cheers,
Michael Larabel
Below is the official change-log between Phoronix Test Suite 1.2 and 1.4 Alpha 1, which is about 50 changes. Here though are the most important ones:
- In addition to having tests and suites stored within their conventional locations (phoronix-test-suite/pts/ or /usr/share/phoronix-test-suite/), they can now be stored within ~/.phoronix-test-suite/test-profiles/ / ~/.phoronix-test-suite/test-resources/ and ~/.phoronix-test-suite/test-suites/ respectively. This makes it easy for a user to install a new or updated test without needing root access and it allows for the easy installation of third-party tests. These directories will also be used for automatically installing some updated tests where available.
- Cascading Test Profiles: This is a new way of writing test profiles that would be similar to the style of Cascading Style Sheets or inheritance in many programming languages. A test profile is now able to use an XML tag of "Extends"that makes it extend another test. For example, all of the WINE-based tests are going to extend a generic WINE test profile that all it does is build the WINE release and sets up the environment. Then all of the other WINE test profiles don't need to worry about this process.Cascading Test Profiles can also be used for sharing other common resources between tests. In Orkdal Alpha 1 there are three base tests currently being extended, for the timed audio encoding, timed build, and video tests. Due to Cascading Test Profiles, the pts-shared test resource has been dropped in this release.
- Self-Contained Test Profiles: This feature allows an entire test profile with test resources to be bundled into a single SCTP file. In doing so, it makes it very easy to distribute small tests. For instance, if you notice a bug with some program and are able to automate it with a Phoronix Test Suite profile, you can write a simple test and throw it into an SCTP file. Following that, just attach it to the BugZilla or what have you. Then any user with the Phoronix Test Suite installed just needs to run./test-name.sctp and it will automatically initialize the Phoronix Test Suite and begin running the test. Self-Contained Test Profiles makes testing very portable and is really intended for just small tests that in many cases would be temporary and are not intended for submission into the Phoronix Test Suite program itself. The Self-Contained Test Profiles feature is implemented, but with only limited testing thus far and needs some documentation on how to write a SCTP.
- The Phoronix Test Suite temp directory is no longer used when downloading files.
- A World of Padman test has been added (named: padman).
There's a lot of other changes in this release, but these are the main changes exposed to the end-user within Alpha 1.
Download: http://www.phoronix-test-suite.com/d...-suite-1.4.0a1
Phoronix Test Suite 1.4.0 Alpha 1
September 18, 2008
- pts-core: Change test/suite type strings to define statements
- pts-core: Add test type caching support
- pts-core: Create the unified PTS test/suite type handler
- pts-core: Clean-up some test installation functions
- pts-core: Provide new architecture for test/suite checking through the type handler
- pts-core: Drop PTS_USER_DIR usage
- pts-core: Add support for having tests and suites stored locally within ~/.phoronix-test-suite/ as an option for users installing new tests/suites or third-party files
- pts-core: Add support so that a test profile can extend another test, for Cascading Test Profile support. This is similar to inheritance in programming languages or Cascading Style Sheets.
- pts-core: Add in support for Self-Contained Test Profiles (SCTP) that initiate the Phoronix Test Suite and runs a stand-alone test
- pts-core: A test profile isn't real: likely incomplete or just a base test within a Self-Contained Test Profile extension
- pts-core: Report identifiers of test profiles being extended to the test profiles being run
- pts-core: Drop PTS shared download support for test profiles, instead they should be using the Cascading Test Profiles functionality
- pts-core: Don't allow the user to save a test result file if it's the same name as a real test profile or test suite
- pts-core: Add Mac OS X operating system detection support (using the IS_MACOSX define)
- pts-core: Improve VirtualBox virtualization detection for guest operating systems running
- pts-core: Add $COMPILER and $VIDEO_DRIVER user variables when saving the results and supplying the test identifier
- pts-core: Let the size of files being downloaded be specified individually within the test's downloads.xml profile instead of a generic one through the test profile XML file
- pts-core: When downloading files for tests, don't first download the file to the PTS temporary directory, but just rename it initially within the test's environment directory
- pts-core: When running a test (or suite that contains a test) on an unsupported architecture or platform, don't abruptly exit
- pts-core: Add module_setup() to pts_module_interface for returning an array of objects that can be configured by the end-user
- pts-core: Add support for persistent configuration options that end-users can easily adjust for PTS modules by running "phoronix-test-suite module-setup MODULE-NAME" that are stored to ~/.phoronix-test-suite/modules-config.xml
- pts-core: Add pts_module::set_option() function to allow a module itself to set a persistent configuration option
- pts-core: Add PTS_QUIT module return type for having the Phoronix Test Suite stop immediately and prematurely (this should only be used in the event of reaching a fatal bug or other problem)
- pts-core: Have the Phoronix Test Suite check once a day to see if the version in use is the latest version, otherwise inform the user of an available update
- pts-core: Add fall-back support for reading the PCI subsystem vendor if the motherboard vendor can't be determined using the other HAL information
- pts-core: Add a PreInstallMessage tag for test profiles that will be shown to the user prior to installing the test
- pts-core: Don't have PreRunMessage or PreInstallMessage hang when in batch mode
- pts-core: pts_find_file() -> pts_find_result_file()
- pts-core: Add initial support for detecting the motherboard Southbridge where applicable
- pts-core: Omit OEM in read_hal()
- pts-core: With list-tests, don't list tests that aren't compliant with the running system's architecture and platform
- pts-core: Add support so test options can be set as an environment variable by using <TEST NAME UPPER_CASE>_<NUM OPTION>=<INDEX OF REQUEST>
- pts-core: Switch to using php_uname() in pts-init.php
- pts-core: Add support for reading all ATI forced anti-aliasing types: Box, Narrow-tent, Wide-tent, and Edge-detect
- pts: Switch all audio encoding tests to extend a timed-audio-encode base test instead of all tests having some of the same code copied throughout
- pts: Switch all build tests to extend a timed-build base test instead of all tests having some of the same code copied throughout
- pts: Switch all video tests to extend a video-sample base test instead of all tests having some of the same code copied throughout
- pts: Change all current and relevant test profiles shipping the Phoronix Test Suite to using the FileSize tags within their downloads.xml files
- pts: Improve the reported environment size for many of the test profiles
- pts: Update compress-7zip test profile to use p7zip 4.58
- pts: Add World of Padman test profile (named padman)
- pts: Add padman to ioquake3-games test suite
- pts: Update Mandriva Linux packages XML file (Thanks To St?phane T?letch?a)
- tandem_Xml: Add a pts_test_tandem_XmlReader object that extends tandem_XmlReader but takes advantage of the handleXmlZeroTagFallback
- tandem_Xml: Add fallback function handler support when a tag isn't found
- graphics_event_checker: Add graphics_event_checker module that makes sure the graphics system isn't messed up or having unintended options enabled while tests are running
- graphics_event_checker: Merge the gpu_error_counter module functionality into the graphics_event_checker module
- timed_screenshot: A module that takes a screenshot while tests are running at a pre-defined interval
- documentation: Add basic documentation concerning the Self-Contained Test Profiles (SCTP) XML tags that are added
Cheers,
Michael Larabel