Originally posted by Michael
View Post
I agree with you that PTS shouldn't do too much post-processing, and leave this responsibility to the app.
It's also true that most games don't provide detailed frame-time measurements,
and therefore I agree that it isn't worth the effort to do anything on the PTS side.
But...
What if practically every game could offer these measurements in the same way, and also do the post-processing such that PTS only needs to parse the resulting jitter metric as one output value?
You probably know what I'm referring to: libframetime[*]
However, currently there are some problems with using libframetime to achieve reproducible and comparable benchmark results:
- not all games support to benchmark right away, such that the first few frames must be left out; this amount of left-out frames could be non-deterministic, making it harder to compare results of different runs
- results are only valid for comparison when the game engine renders independent of local time, i.e., it should use a fixed simulation step / delta game-time between each frame
Let me try to clarify the last point with the following simplified pseudo code:
Conventional game loop (without VSync of course), not suitable for benchmarking:
Code:
gameTime = gameTimeEpoch previousTime = time() while True: currentTime = time() deltaTime = currentTime - previousTime gameTime += deltaTime render(scene(gameTime)) previousTime = currentTime
Code:
gameTime = gameTimeEpoch while True: deltaTime = 1.0 / 60 # as if game was rendered @ 60 fps gameTime += deltaTime render(scene(gameTime))
Let's suppose we're not allowed to modify the source code of the game to implement a proper benchmarking game loop,
then we can just create a hook for time() (and render()) to accomplish this for (nearly) every game:
Code:
time_orig = time # backup of real time() function render_orig = render # backup of real render() function (actually glXSwapBuffers()) currentTime = time_orig() def time(): return currentTime def render(): render_orig() currentTime += 1.0 / 60 # as if game was rendered @ 60 fps
- gl-117
- 0ad
- supertuxkart
- flightgear
- xonotic (localhost of course)
Visual proof can be found here.
If I would add this functionality to libframetime, and create the script to generate the jitter metric, and maybe also create some test profiles,
would you possibly reconsider your decision and sometimes feature the new jitter metric in some of your future articles?
.[*]: a new post-processing script can be created to use the frame-times file as input and output the jitter metric, e.g., something like this awk script
Leave a comment: