Help with parsing PTS sockperf "# dropped messages" results

[JesseG]

I am using the PTS/sockperf under-load test and am trying to figure out how to parse / return the number of dropped messages result from the ouput.

Here is what sockperf output looks when run "by hand":
[root@HOSTNAME sockperf-3.4]# ./sockperf server &
[1] 55377
[root@HOSTNAME sockperf-3.4]# sockperf: == version #3.4-no.git ==
sockperf: [SERVER] listen on:
[ 0] IP = 0.0.0.0 PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: [tid 55377] using recvfrom() to block on socket(s)

[root@HOSTNAME sockperf-3.4]# ./sockperf under-load -i 127.0.0.1 -p 11111 -t 10 -m 64 --pps=max
sockperf: == version #3.4-no.git ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s)

[ 0] IP = 127.0.0.1 PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=10.000 sec; Warm up time=400 msec; SentMessages=2882725; ReceivedMessages=28827
sockperf: ========= Printing statistics for Server No: 0
sockperf: Test end (interrupted by signal 2)
sockperf: [Valid Duration] RunTime=9.550 sec; SentMessages=2754401; ReceivedMessages=27545
sockperf: ====> avg-lat= 17.551 (std-dev=0.853)
sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
sockperf: Summary: Latency is 17.551 usec
sockperf: Total 27545 observations; each percentile contains 275.45 observations
sockperf: ---> <MAX> observation = 30.960
sockperf: ---> percentile 99.999 = 30.960
sockperf: ---> percentile 99.990 = 29.451
sockperf: ---> percentile 99.900 = 27.449
sockperf: ---> percentile 99.000 = 20.403
sockperf: ---> percentile 90.000 = 18.092
sockperf: ---> percentile 75.000 = 17.754
sockperf: ---> percentile 50.000 = 17.456
sockperf: ---> percentile 25.000 = 17.210
sockperf: ---> <MIN> observation = 5.136

The line I wish to parse is the following:
sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0

So, I modified results-definition.xml, adding the following stanza:
<ResultsParser>
<OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
<ResultScale>Number of Dropped Messages</ResultScale>
<ResultProportion>LIB</ResultProportion>
</ResultsParser>

However, this does not succeed in parsing out the desired value. Here is the relevant output of "phoronix-test-suite debug-run sockperf":
Result Key: #_RESULT_#
Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
Result Parsing Search Key: "sockperf: # dropped messages ="
Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
No Test Results

It looks to me as if the ";" that follows directly after the result is being parsed and so the resulting string ("0;") does not match #_RESULT_#. I have tried adding ResultBeforeString to the stanza with no luck. Is there some directive (ResultAfterString did not work) that applies here? I would hope this is a condition that has been seen before and for which there is a PTS parsing directive.

Please advise, thanks in advance,
Jesse

[Michael]

Hi, does it work if you add <StripFromResult>;</StripFromResult> ? Otherwise I can try playing with it this afternoon as it may be a useful secondary metric to add to the test profile upstream.

[JesseG]

Michael,

Thanks for the prompt reply. I updated results-definition.xml as you suggested:
<ResultsParser>
<OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
<StripFromResult>;</StripFromResult>
<ResultScale>Number of Dropped Messages</ResultScale>
<ResultProportion>LIB</ResultProportion>
</ResultsParser>

However, this did not seem to work.
Result Key: #_RESULT_#
Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
Result Parsing Search Key: "sockperf: # dropped messages ="
Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
Test Result Parser Returning: 0

I also tried removing the semi-colons from the OutputTemplate line (to possibly match the StripFromResult outcome) but that did not work, either.

Thank you.

[Michael]

Michael,

Thanks for the prompt reply. I updated results-definition.xml as you suggested:
<ResultsParser>
<OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
<StripFromResult>;</StripFromResult>
<ResultScale>Number of Dropped Messages</ResultScale>
<ResultProportion>LIB</ResultProportion>
</ResultsParser>

However, this did not seem to work.
Result Key: #_RESULT_#
Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
Result Parsing Search Key: "sockperf: # dropped messages ="
Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
Test Result Parser Returning: 0

I also tried removing the semi-colons from the OutputTemplate line (to possibly match the StripFromResult outcome) but that did not work, either.

Thank you.

This does work:

  <ResultsParser>
    <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
    <LineHint>dropped messages</LineHint>
    <StripFromResult>;</StripFromResult>
    <ResultScale>Dropped Messages</ResultScale>
    <ResultProportion>LIB</ResultProportion>
  </ResultsParser>

I verified it via the debug-result-parser command.

But if the number of dropped messages is 0, it won't show the result because that is the default behavior. Were you trying when you did have dropped messages?

[JesseG]

Michael, Ah, most interesting. I actually had the right idea :-), as you posted: <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate> <StripFromResult>;</StripFromResult> I guess I saw the line at the end of the output "The test run did not produce a result." but missed the change from: No Test Results to: Test Result Parser Returning: 0 You are correct in that I did not have any dropped messages. I was trying this all out locally first to get it all working, before attempting to deploy in an environment where dropped messages may occur. What I am hoping to do is to track dropped messages over time. For that purpose, 0 is a valid result. So, the new question is how to understand / modify the default behavior so that the 0 results can be returned as a valid test result?

[Michael]

Michael, Ah, most interesting. I actually had the right idea :-), as you posted: <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate> <StripFromResult>;</StripFromResult> I guess I saw the line at the end of the output "The test run did not produce a result." but missed the change from: No Test Results to: Test Result Parser Returning: 0 You are correct in that I did not have any dropped messages. I was trying this all out locally first to get it all working, before attempting to deploy in an environment where dropped messages may occur. What I am hoping to do is to track dropped messages over time. For that purpose, 0 is a valid result. So, the new question is how to understand / modify the default behavior so that the 0 results can be returned as a valid test result?

I'll look at adding a new option along the lines of <ZeroValueIsValid>TRUE</ZeroValueIsValid> or the likes. At this point it's been assumed and default behavior where if value is 0, not to show it so would make it a new PTS option to handle this use-case.