As some of you are aware, The Tech Report have been using a new type of analysis on FPS to analyse frame latencies (see http://techreport.com/review/21516/i...e-benchmarking).
To do this yourself I have devised a methodology (which could be automated) using Doom 3.
Step 1:
Record frame timings (I have used the command from the OpenBenchmarking Doom3 Test Profile). The important bit is the com_speeds flag:
Step 2:
Parse log file using this Python script:
Step 3:
Use this R script on resulting csv file:
Example of resulting images:
To do this yourself I have devised a methodology (which could be automated) using Doom 3.
Step 1:
Record frame timings (I have used the command from the OpenBenchmarking Doom3 Test Profile). The important bit is the com_speeds flag:
Code:
./doom3 +exec doom3-pts.cfg +set sys_VideoRam 512MB +set r_mode -1 +timedemoquit demo1 +set com_speeds 1 > doom3.log
Parse log file using this Python script:
Code:
import re # Compile reg exp re1='.*?' # Non-greedy match on filler re2='(\\d+)' # Integer Number 1 r = re.compile(re1+re2+re1+re2+re1+re2+re1+re2+re1+re2,re.IGNORECASE|re.DOTALL) # Open log file frames = [] f = open('doom3.log') for line in f.readlines(): if line.startswith('frame:'): m = r.search(line) if m: frames.append( {'com_frameNumber': int(m.group(1)), 'com_frameMsec': int(m.group(2)), 'time_gameFrame': int(m.group(3)), 'time_frontend': int(m.group(4)), 'time_backend': int(m.group(5))} ) f.close() # Write to data file f = open('data.csv', 'w') f.write('Frame,FrameTime\n') for frame in frames: # Skip first frame if frame['com_frameNumber'] == 0: continue f.write('%i,%i\n' % (frame['com_frameNumber'], frame['com_frameMsec'])) f.close()
Use this R script on resulting csv file:
Code:
dat <- read.csv("data.csv",header=T,colClasses=c("integer","numeric")) summary(dat) # Progression of frame times plot(dat, type='n', xlab="Frame number", ylab="Frame time in ms (lower is better)") lines(dat) # Average fps max(dat$Frame)*1000/sum(dat$FrameTime) # Percentiles # It's the point below which 99% of all frames have been rendered. # We're simply excluding the last 1% of frames, many of them potential outliers, # to get a sense of overall smoothness. quantile(dat$FrameTime, .99) # Frame latencies by percentile p <- seq(0, 1, length.out=1000)^(1/3) quan <- data.frame(q = quantile(dat$FrameTime, probs = p), prob = p) plot(quan$prob, quan$q, type='n', xaxt='n', xlim=c(0.5,1), xlab="Proportion of frames rendered", ylab="Frame time in ms (lower is better)") axis(1, at=seq(0,1,by=.05), labels=paste(100*seq(0,1,by=.05), "%") ) lines(quan$prob, quan$q) # Time spent beyond 50 ms sum(subset(dat,FrameTime>50)$FrameTime) # Time spent beyond 16.7 ms sum(subset(dat,FrameTime>16.7)$FrameTime)
Example of resulting images:
Comment