Announcement

Collapse
No announcement yet.

Help with parsing PTS sockperf "# dropped messages" results

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Help with parsing PTS sockperf "# dropped messages" results

    I am using the PTS/sockperf under-load test and am trying to figure out how to parse / return the number of dropped messages result from the ouput.

    Here is what sockperf output looks when run "by hand":
    [[email protected] sockperf-3.4]# ./sockperf server &
    [1] 55377
    [[email protected] sockperf-3.4]# sockperf: == version #3.4-no.git ==
    sockperf: [SERVER] listen on:
    [ 0] IP = 0.0.0.0 PORT = 11111 # UDP
    sockperf: Warmup stage (sending a few dummy messages)...
    sockperf: [tid 55377] using recvfrom() to block on socket(s)

    [[email protected] sockperf-3.4]# ./sockperf under-load -i 127.0.0.1 -p 11111 -t 10 -m 64 --pps=max
    sockperf: == version #3.4-no.git ==
    sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s)

    [ 0] IP = 127.0.0.1 PORT = 11111 # UDP
    sockperf: Warmup stage (sending a few dummy messages)...
    sockperf: Starting test...
    sockperf: Test end (interrupted by timer)
    sockperf: Test ended
    sockperf: [Total Run] RunTime=10.000 sec; Warm up time=400 msec; SentMessages=2882725; ReceivedMessages=28827
    sockperf: ========= Printing statistics for Server No: 0
    sockperf: Test end (interrupted by signal 2)
    sockperf: [Valid Duration] RunTime=9.550 sec; SentMessages=2754401; ReceivedMessages=27545
    sockperf: ====> avg-lat= 17.551 (std-dev=0.853)
    sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
    sockperf: Summary: Latency is 17.551 usec
    sockperf: Total 27545 observations; each percentile contains 275.45 observations
    sockperf: ---> <MAX> observation = 30.960
    sockperf: ---> percentile 99.999 = 30.960
    sockperf: ---> percentile 99.990 = 29.451
    sockperf: ---> percentile 99.900 = 27.449
    sockperf: ---> percentile 99.000 = 20.403
    sockperf: ---> percentile 90.000 = 18.092
    sockperf: ---> percentile 75.000 = 17.754
    sockperf: ---> percentile 50.000 = 17.456
    sockperf: ---> percentile 25.000 = 17.210
    sockperf: ---> <MIN> observation = 5.136

    The line I wish to parse is the following:
    sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0

    So, I modified results-definition.xml, adding the following stanza:
    <ResultsParser>
    <OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
    <ResultScale>Number of Dropped Messages</ResultScale>
    <ResultProportion>LIB</ResultProportion>
    </ResultsParser>

    However, this does not succeed in parsing out the desired value. Here is the relevant output of "phoronix-test-suite debug-run sockperf":
    Result Key: #_RESULT_#
    Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
    Result Parsing Search Key: "sockperf: # dropped messages ="
    Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
    No Test Results

    It looks to me as if the ";" that follows directly after the result is being parsed and so the resulting string ("0;") does not match #_RESULT_#. I have tried adding ResultBeforeString to the stanza with no luck. Is there some directive (ResultAfterString did not work) that applies here? I would hope this is a condition that has been seen before and for which there is a PTS parsing directive.

    Please advise, thanks in advance,
    Jesse

  • Michael
    replied
    Originally posted by JesseG View Post
    Michael, Ah, most interesting. I actually had the right idea :-), as you posted: <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate> <StripFromResult>;</StripFromResult> I guess I saw the line at the end of the output "The test run did not produce a result." but missed the change from: No Test Results to: Test Result Parser Returning: 0 You are correct in that I did not have any dropped messages. I was trying this all out locally first to get it all working, before attempting to deploy in an environment where dropped messages may occur. What I am hoping to do is to track dropped messages over time. For that purpose, 0 is a valid result. So, the new question is how to understand / modify the default behavior so that the 0 results can be returned as a valid test result?
    I'll look at adding a new option along the lines of <ZeroValueIsValid>TRUE</ZeroValueIsValid> or the likes. At this point it's been assumed and default behavior where if value is 0, not to show it so would make it a new PTS option to handle this use-case.

    Leave a comment:


  • JesseG
    replied

    Michael, Ah, most interesting. I actually had the right idea :-), as you posted: <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate> <StripFromResult>;</StripFromResult> I guess I saw the line at the end of the output "The test run did not produce a result." but missed the change from: No Test Results to: Test Result Parser Returning: 0 You are correct in that I did not have any dropped messages. I was trying this all out locally first to get it all working, before attempting to deploy in an environment where dropped messages may occur. What I am hoping to do is to track dropped messages over time. For that purpose, 0 is a valid result. So, the new question is how to understand / modify the default behavior so that the 0 results can be returned as a valid test result?

    Leave a comment:


  • Michael
    replied
    Originally posted by JesseG View Post
    Michael,

    Thanks for the prompt reply. I updated results-definition.xml as you suggested:
    <ResultsParser>
    <OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
    <StripFromResult>;</StripFromResult>
    <ResultScale>Number of Dropped Messages</ResultScale>
    <ResultProportion>LIB</ResultProportion>
    </ResultsParser>

    However, this did not seem to work.
    Result Key: #_RESULT_#
    Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
    Result Parsing Search Key: "sockperf: # dropped messages ="
    Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
    Test Result Parser Returning: 0

    I also tried removing the semi-colons from the OutputTemplate line (to possibly match the StripFromResult outcome) but that did not work, either.

    Thank you.
    This does work:

    Code:
      <ResultsParser>
        <OutputTemplate>sockperf: # dropped messages = #_RESULT_# # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
        <LineHint>dropped messages</LineHint>
        <StripFromResult>;</StripFromResult>
        <ResultScale>Dropped Messages</ResultScale>
        <ResultProportion>LIB</ResultProportion>
      </ResultsParser>
    I verified it via the debug-result-parser command.

    But if the number of dropped messages is 0, it won't show the result because that is the default behavior. Were you trying when you did have dropped messages?

    Leave a comment:


  • JesseG
    replied
    Michael,

    Thanks for the prompt reply. I updated results-definition.xml as you suggested:
    <ResultsParser>
    <OutputTemplate>sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0</OutputTemplate>
    <StripFromResult>;</StripFromResult>
    <ResultScale>Number of Dropped Messages</ResultScale>
    <ResultProportion>LIB</ResultProportion>
    </ResultsParser>

    However, this did not seem to work.
    Result Key: #_RESULT_#
    Template Line: sockperf: # dropped messages = #_RESULT_#; # duplicated messages = 0; # out-of-order messages = 0
    Result Parsing Search Key: "sockperf: # dropped messages ="
    Result Line: sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
    Test Result Parser Returning: 0

    I also tried removing the semi-colons from the OutputTemplate line (to possibly match the StripFromResult outcome) but that did not work, either.

    Thank you.

    Leave a comment:


  • Michael
    replied
    Hi, does it work if you add <StripFromResult>;</StripFromResult> ? Otherwise I can try playing with it this afternoon as it may be a useful secondary metric to add to the test profile upstream.

    Leave a comment:

Working...
X