christian_lappy: still no steam4linux ..zzz
christian_lappy: morning michaellarabel
michaellarabel: Hi christian_lappy
christian_lappy: a wine perf test over different kernels would be fantastic....
christian_lappy: or different distros
michaellarabel: I need more Wine tests in PTS first.
christian_lappy: mhhh
christian_lappy: michaellarabel: same nv+ati test with diff kernels ?
michaellarabel: Not sure what good that would do.
christian_lappy: see if kernel matters game perf ?
dandel: hey michaellarabel, i got a superpi replacement test almost set for use on pts, the only problem i am having is parsing the output.
michaellarabel: dandel: What problem are you having with parsing the output?
dandel: i can't pull the output properly.
dandel: it has spaces
michaellarabel: What does the output look like?
dandel: http://pastebin.ubuntu.com/472224/ (This)
michaellarabel: which part are you trying to extract or use?
dandel: 16 128K N/A 0h 0m 0.550s < this
dandel: the line just before the checksum
dandel: that is the result for the pi calculation
dandel: also, borweign is fast when calculating smaller digits, but with 8m it takes 12+ minutes
michaellarabel: I am not sure if a test that only takes a half-second to run though would be so good?
dandel: oh, that's 128k digits
dandel: the test can be up to 128M digits
michaellarabel: oh okay, well one second and I'll write you up a sample parser interface.
dandel: 20 1M N/A 0h 0m 3.634s < 1m digits with the snappy Gauss algorithm
michaellarabel:
michaellarabel:
michaellarabel:
michaellarabel: 128k
michaellarabel: 16 128K N/A 0h 0m #_RESULT_#
michaellarabel: s
michaellarabel:
michaellarabel:
michaellarabel: errr lemme pastebin this
dandel: no good.
dandel: the test uses 0h and 0m as part of the results
dandel: forces a result which is going to take a few min.
michaellarabel: http://pastebin.ubuntu.com/472226/ here is the clean version
michaellarabel: oh yeah, hmmm
michaellarabel: are you building the test from source?
dandel: http://systester.sourceforge.net/ (Source)
dandel: yes.
dandel: ^.^ it's a prime candidate for inclusion, the only dependency is libgmp
michaellarabel: If the test is being built from source, the very easiest way may be to just patch the program to not break up the h, m, and s reporting.
dandel: Sample test with this: /systester-cli -bench -qcborwein 8M
dandel: that takes a lot of time
dandel: anyways, part of the test is the fact that this is 100% threaded.
dandel: you can specify up to 64 threads.
dandel: gausslg is about the same as super pi on result times, and the Borweign can be called a cpu torture due to how long it takes.
michaellarabel: k
dandel: http://pastebin.ubuntu.com/472228/ (Output that clearly shows what i mean)
dandel: 8M digits on borwein is about 3 minutes, while the gauss is at about 1 minute.
dandel: as thus, this is pretty much a 100% drop in replacement for super pi, but has a few more features.
dandel: if i was a bit better at bash scripting, i could combine the output using sed and expr.
dandel: hmm... it'd be nice if they used svn or git ><; (makes generating the patches much easier.)
IgnisInCaelum: Boo.
dandel: michaellarabel, i finished modifying the code, and now all i need is to figure out what to do about generating the patch that will be downloaded with the test.
michaellarabel: If you need the patch hosted I can host it.
michaellarabel: Or if its small, just embed it within the install.sh file
dandel: the patch is small enough to be embedded within install.sh
dandel: the change is less than 20 lines of code.
michaellarabel: k
dandel: to change the output to: 24 16M N/A 162.878s
michaellarabel: that previous XML I pasted should be able to be modified easily to do that with copying and pasting
xlemming: hi michaellarabel, I have a question about test-profiles and parsing their results: is there a way to parse multiple results from a single test? for instance, in a normal ramspeed test it runs the copy, scale, add, and triad tests each time, but in order to to get results for each type of test (copy, scale, add, triad) it looks like pts runs the same ramspeed test multiple times. is there a way to have it run the test once, but have it parse
michaellarabel: Yes, if you look at the latest Git code there are a few examples... Look at like scimark2, ramspeed (I think RAMspeed is run that way because upstream there is no way to just have it run only the copy operation or something), or blogbench.
michaellarabel: Though actually, I see what you are saying now about it in regards to running... There is a way to better handle this, yes.
xlemming: excellent, thanks. would i be able to find the same sort of examples in 2.6 or 2.8, or just the latest git code?
dandel: hmm... I can't seem to get the patch to apply.
michaellarabel: xlemming: The latest Git is for 2.8 and between 2.6 and 2.8 there were some changes to some test profiles and I just don't remember if I changed ramspeed pre-2.6 or in this pre-2.8 cycle, so I say Git just to be safe :) If you guys are still on 2.6 try adding: TRUE after the tag (within ) of ramspeed.xml and that may be all you need.
michaellarabel: The AllowCacheShare allows the test result outputs to be cached on that run if the given test is set to be run multiple times and the upstream test ends up running through all tests each time like RAMspeed does with always running Copy/Scale/Add/Triad regardless.
xlemming: sounds good enough, I will try that out, it should streamline things a bit
michaellarabel: I'm surprised though I didn't add that option before to ramspeed.xml unless I just overlooked it before. But if you have any problems with it let me know and I'll run it on one of my systems to verify.
xlemming: not sure, i was working off of 2.6.0, so I will take a look at later version too
dandel: michaellarabel, the parser you recommended looks like it will always pull the wrong result.
michaellarabel: xlemming: I am checking against Git right now and the AllowCacheShare isn't in there.
michaellarabel: dandel: What's it pulling?
dandel: on the 4m test it will pull: 17 128K N/A 18.411s
dandel: when it should pull: 22 4M N/A 23.345s
xlemming: alright, thanks for the info, I will try to add that argument as you described and see if it works then
michaellarabel: dandel: can you pastebin that output plus your current parser?
michaellarabel: xlemming: No problem, email me if it doesn't work or if you hit any other problems.
dandel: ok... one sec.
dandel: i'm double checking with various options to guarantee.
dandel: the only thing that needs to be added to the system test information is the version of libgmp that is installed.
dandel: http://pastebin.ubuntu.com/472259/
dandel: what the parser needs to do is query the identifier i set as Digits for the line hint.
michaellarabel: won't the digits count always be the same?
dandel: no.
michaellarabel: Under what cases are you changing it?
dandel: to vary the test results.
dandel: i have test conditions from 1m to 32m in multiples of 2.
dandel: the line that should be parsed is always the line just before it says checksum validation.
michaellarabel: Is there much value to that though? Since I think most users would just arbitrarily pick some number and not give it much thought so having something more consistent would be of use to more of the normal users.
dandel: i based what i did on the super-pi from 2.6.x
dandel: this is pretty much a drop in replacement for super-pi for the performance and times for calculation.
michaellarabel: There is a LineBeforeHint for indicating the line before it, but there is no LineAfterHint for saying "Checksum", but I can add such an option... I just personally think though having one set number right in the middle will be of more value.
dandel: the more options are generally a good idea to show a performance curve.
dandel: after all, i even have the ability to control the number of threads.
michaellarabel: Right, in theory it's good, I am just not sure how many people will actually take advantage of it though... Especially with Phoronix Global comparisons and such if they all end up picking a number nearly at random for how long they want to test that those sorts of users will not have much value to the results
dandel: hmm...
dandel: i'll reduce the options to 3.
michaellarabel: k
dandel: 4m, 16m and 64M
michaellarabel: I'll add a LineAfterHint tag right now so you could just specify Checksum validation in place of
dandel: ok.
dandel: what about thread option tho... i have that with a ton of options also.
dandel: i have that set to 1, 2, 3, 4, 8, 16, 32, and 64.
michaellarabel: why not just max it out for the CPU count or twice the CPU count?
dandel: hmm?
michaellarabel: At least with the other multi-threaded tests, most users will just use the default for fully maxing their system whether the optimal number of threads is the thread count of the system or 2x the thread count.
dandel: test results... it's no good to compare the 2 threads vs 4 threads.
michaellarabel: If you are comparing the hardware performance though it normally is.
dandel: hmm... is there a warning flag that i could add to the menu option telling the user which test could take insane ammounts of time?
michaellarabel: Such an option can be added if you can explain what you envision for it.
dandel: Borwein's Quadratic Convergence is good for benchmarks, but takes a long time to run at higher digit counts
dandel: for example: 4 threads and 8M digits will take 12 minutes to complete easily on a quad core with 2.8ghz
michaellarabel: to just append that to the string when showing the options or what though?
dandel: when showing the option, or when the test is running.
michaellarabel: well when the test is running it's too late.
dandel: that way the user is informed that the test is working, but could take a long time.
dandel: it could be in the test time or something.
dandel: ><; actually, when i use the borwein, 3 tests on that are too much because of the fact it could easily run into the hours mark
michaellarabel: One second, let me finish up this LineAfterHint support and then I can figure out what'd be the best for the warning message.
dandel: other tests could warrant this warning depending on what happens.
dandel: if i run the test with the options: -qcborwein 4M -bench -threads 8 it takes 118s just to get to the first 32k digits.
michaellarabel: http://phorogit.com/index.php?p=phoronix-test-suite.git&a=commitdiff&h=e041e76265e2c7980a31e0e9769ef64c82c0e44b&hb=e9e4e1bd12b0be178e4f1cc8745a52abe5d20a94 should provide the LineAfterHint support if you just want to grab the latest Git
dandel: just changing the algorithm used is enough to warrant the warning: 169.654s (Borwein) and 52.023s (gauss)
michaellarabel: well how would you be calculating this warning? I thought you were just going to bind it to one of the option names?
dandel: I am, for now.
dandel: I just tacked on to the name: (Warning: Very long test)
dandel: the only thing i have left is to add the libgmp installation and detection. (since this will have a direct influence on test results)
zaib__: hi i have a old laptop http://paste.ubuntu.com/472258/ plz tell me that will lubuntu run faster xp on my laptop
zaib__: plz help
dandel: hmm... now it says test did not produce a result 0o'
michaellarabel: dandel: Running Git? Try running: debug-run systester
dandel: oh has results, but it's the wrong result
dandel: it's pulling 4m instead of the time
michaellarabel: is it going two lines back then or what?
dandel: it has the right line
dandel: it's pulling the wrong part tho
dandel: 22 4M N/A 52.023s (on this string, it's grabbing 4M instead of 52.023s)
michaellarabel: hmm, try setting the 16 128K N/A #_RESULT_# as like 16 BLAH 128K N/A #_RESULT_# or anything else to try to offset it to figure out where the parsing is not liking it.
dandel: I'm looking into what is happening.
dandel: hmm... 16m digits is what some would call too long 0o'
dandel: 3m on the quick algorithm ><;
dandel: ok, it seems the results are more or less accurate now.
michaellarabel: Okay.
dandel: 16m digits = 149.4s when using gausslg in 4 threads.
dandel: and 4m digits = 26.63s with gausslg in 4 threads
michaellarabel: With the very latest Git code, you can now have a String tag within just like the Name and Value tags for anything you want shown to the user when displaying that list of options.
dandel: that's good... now i can add warnings about runtime to the larger tests taking a lot longer to run.
dandel: the message line is really useful... i'll put a short message on the tests to make it easier so users know which test to run.
michaellarabel: k
dandel: 32 and 64 threads mentioning server in the message should be enough.
dandel: and now all that is left is to add the distribution specific parts (at which i have not looked into)
dandel: michaellarabel, where should i send the git patch to?
michaellarabel: michael at phoronix.com works
dandel: The only part that is remaining is to make sure libgmp headers and dev libraries get installed as the distribution specific stuff.
michaellarabel: isn't there already an external dependency on that?
dandel: no
dandel: not that i know of
michaellarabel: Hmm okay, I thought there was, but if not I can easily add that in.
dandel: i have it dependent on build tool
dandel: patch sent
dandel: actually, it might be possible to host the systester on phoronix directly (The file source file under 200kb)
dandel: * the file that contains the source is under
michaellarabel: Any reasoning for naming it proc-systester rather than just systester?
dandel: i thought another tool was called systester, and i didn't want to confuse the two.
michaellarabel: number one hit in google for systester is this one you refer to
dandel: ah
michaellarabel: would you prefer it be proc-systester though?
dandel: not really.
michaellarabel: k, I'll rename it to systester then
dandel: i don't care... anyways, this test can have a windows binary done so ya know.
dandel: i won't bother fixing it, but as long as you take care of the libgmp requirement, adding the binaries for windows is easy.
michaellarabel: Okay, it should be in Git now as systester. Thanks a lot! Will add the gmp external dependency this afternoon
dandel: this makes a return of the pi tests everyone is so fond of ^.^
dandel: michaellarabel, i thought of a good idea... why not add a test/suite uninstall option to pts (The help menu says that there is no way to tell phoronix-test-suite what to remove.)
dandel: oh and i'm currently running the new test so i can put the first result showing why it's important to read the user message/warning. (by doing all of the options on threads using gauss-legendre algorithm to 4m digits)
dandel: 16 threads is not too bad on time: avg runtime of 102.28 Seconds
dandel: i expect 32 threads to be about 204 seconds and for 64 threads to be about 416 seconds on average.
xlemming: hi michaellarabel (not sure if you're still around): so, adding the TRUE tag almost works. However (using ramspeed as an example again), this causes it to iterate the same results through all the test options. In the ramspeed example, it uses those results from the first integer test for all the tests. This is because the results parser for ramspeed matches only the COPY/SCALE/ADD arguments, and re-running
michaellarabel: xlemming: So when switching between floating point and integer is where you're having the problem?
xlemming: well switching between any other test options
xlemming: when using that tag, it reiterates through the results for every test variation
dandel: michaellarabel, i went ahead and put up an test result using the systester, and so far all that is needed is other results to compare against.
xlemming: but for some test options it needs to re-run the test to have the proper results
michaellarabel: xlemming: Ah, okay, I will take a look at it on one of my boxes then to get the cache share to properly support RAMspeed
dandel: 0o' odd... debug-run won't work.
xlemming: alright, its more that i would need to have the test runs use the cached results only when iterating through certain test options, for varying test options it would need to re-run the test
dandel: michaellarabel, the boerwin test with systester on the git is busted because the output is just slightly different than expected :/
dandel: 22 4M N/A 23.094s (gauss) and 21 4M N/A 57.940s (borwein)
michaellarabel: xlemming: Right, I just went back through my notes and this is the reason why AllowCacheShare was disabled for RAMspeed. With the other cache share tests the different options don't mess up the results. But I will extend AllowCacheShare to be able to tell the cache share what option(s) break the output.
dandel: and the only difference is the that gauss has 22 loops at the start and borwein is 21 loops.
xlemming: excellent, looking forward to it
michaellarabel: dandel: That shouldn't matter though whether it says 21 or 22 since the line determination is being done by the line after that
dandel: i know, but it's not picking up the results.
michaellarabel: what does debug-run say?
dandel: won't run
dandel: when i launch phoronix-test-suite debug-run systester it immediately exits
dandel: one sec, while i test the patch
dandel: i think i already got a fix, and it's almost there.
dandel: just need to make sure that it does not break the support for gauss (super-pi).
dandel: http://global.phoronix-test-suite.com/index.php?k=profile&u=defaultuser-31827-6426-9306 (Not bad for initial results)
dandel: it'd help if i could pull the latest git tho... the git clone and pull commands stop at the revision just before systester hit the repo.
dandel: I think it's a good thing you can have multiple parsers in the same xml file (might not be an intended feature, but works)
dandel: http://pastebin.ubuntu.com/472338/ (Parser fix that works, just needs to applied)
dandel: anyways, the more i run these tests, the more accurate the estimated time will be. (since that was completely wrong for most of the tests)
michaellarabel: dandel: So that pastebin just like that works fine?
dandel: yes.
dandel: + the xml tag tho
dandel: simple fix, but works well, and the results for 16m also works, but the borwein tests won't get as much testing... runtime is too long.
dandel: i figure that if i have results working on 4m and 16m for gauss and the borwein uses nearly identical with minor corrections and that works then all is good.
michaellarabel: Interesting that it works like that... I wrote the ability to define multiple parsers in the XML file, but had designed it to be used with some other tags for matching to test selection, etc. But good to know that the parser works just fine in a situation like this.
dandel: this also establishes a bit of regression testing within pts since you know a test which uses this.
michaellarabel: Pushed in Git and refreshed.
dandel: http://global.phoronix-test-suite.com/index.php?k=profile&u=dandel-16489-24764-7729 (latest test result, and the systester is at the bottom)
michaellarabel: If you come up with some good run settings for the relevant suites, I'll happy accept patches there too :)
dandel: oh, that's easy
dandel: with multicore just set use gauss, 4m digits and 64 threads and you got a long haul test for threaded.
dandel: however, that's for 4+ cores, because if you look at my initial results, double the time required on dual cores for 4+ threads.
dandel: odd... ./phoronix-test-suite list-suites says 56 suites, but does not list the actual suites.
dandel: and i know that was working a little while ago.
dandel: http://pastebin.ubuntu.com/472346/ (Regression within pts i spotted)
dandel: the regression is that you see no suites when you run ./phoronix-test-suite list-suites
dandel: michaellarabel, what is the run-time ceiling i should use for the testing?
michaellarabel: there is no ceiling perse
michaellarabel: dandel: Thanks for the note on the regression
michaellarabel: If you want it integrated in a suite though, not 20+ minutes.
dandel: hmm...
dandel: 20+ min on what type of hardware?
dandel: http://pastebin.ubuntu.com/472349/ (This should be a good set of tests)
michaellarabel: medium I say
dandel: the only problem whould be single core machines when the test goes into the Borwein tho.
dandel: however, for a quad core built for under say $800, the longest borwein test would be about 8 min.
dandel: however, if your talking about a dual core, the test will easily take 4x as long.
michaellarabel: Anyways, latest git fixes the list-suites regression. Thanks, was an important regression I made over the weekend that touched a different area of pts-core, but was a simple fix :)
dandel: http://en.wikipedia.org/wiki/Gauss%E2%80%93Legendre_algorithm (the quick algorithm for pi calculation that super-pi uses)
dandel: http://en.wikipedia.org/wiki/Borwein%27s_algorithm (borwein's algorithm that takes a lot of time)
dandel: actually, borwein might be a good test for the computational-biology test since a lot of those use n^2 algorithms normally.
dandel: FliPPeh, you there?
FliPPeh: dandel: sup
dandel: the systester program was added to the phoronix test suite git repo, mind trying that out on your system?
FliPPeh: later
RambJoe: http://forums.gamesforwindows.com/t/13533.aspx
RambJoe: this is interesting
riisen: damn i really want valve to go out and say something bout steamonlinux :/
IgnisInCaelum: ...
IgnisInCaelum: What is this, a troll-kid?
riisen: ofc
IgnisInCaelum: Bleh...
riisen: ehm how can it be troll when there is no topic to start with?
FliPPeh: STEMA???ßßßßßßßßßß
FliPPeh: ON LEENUX?
FliPPeh: jolts awake
riisen: FliPPeh, xD
Hertta: Will neer hpappen
riisen: well it is weird that they want linux coders for no reason in that case..
FliPPeh: They've wanted them for years
Hertta: Just for the fun of it.
riisen: well yea... but for what reason if they dont want anything from them ?
FliPPeh: Server development
IgnisInCaelum: smells burnt riisen
riisen: FliPPeh, sure..
riisen: Port Windows-based games to the Linux platform.
riisen: that part then
riisen: i dont think they wanna hire someone to port it and then dont release it...
riisen: makes no sense
IgnisInCaelum: Oho?
riisen: http://www.valvesoftware.com/job-SenSoftEngineer.html
IgnisInCaelum: realizes that that smell was burnt IgnisInCaelum
riisen: rofl
IgnisInCaelum: dances