Originally posted by phoronix
View Post
I am citing Michal Kubeček [email protected] who has clearly much more knowledge of benchmarking or networks than me. Maybe you can find some valid points there:
[Michael] goes to long details to explain what disks do server and client have but not a word about NICs; "Gigabit Ethernet" is not nearly enough - and that hidden "Realtek PCIe GBE Family + Microsoft ISATAP" (???) isn't much better. [...] No information about how were client and server connected [or] netfilter configuration (which can affect the latency quite a log) and other networking settings [...] the author was running netperf TCP_RR and UDP_RR test (i.e. one very specific aspect of networking performance) with [unknown] parameters [...]
Some tests show variance so high that the results should have been discarded and tests repeated [...] netperf has "-I" and "-i" parameters to make things easier. He runs the same test with two different lengths [...] the results should be the same within a margin of statistic error; if they are not, it should be a warning sign that something was wrong. [...]
One thing that really stands out is the UDP_RR test on Tumbleweed, in particular the 360s one. The variance itself [tells] the test went completely bonkers [...] even upper end of the indicated interval is still way below the results of the 60s test. [...] I quickly ran netperf UDP_RR between my two machines, one running 42.3 with 4.15.8 Kernel:stable kernel (i.e. essentially Tumbleweed), the other Tumbleweed with 4.16-rc4 kernel from Kernel:HEAD. The hardware is definitely worse than Mr. Larabel's and NICs aren't anything special either (on-board Realtek 8168evl and common consumer grade Intel (82541GI)). Both machines are running a KDE desktop (I only wanted to get some idea about the results) so I used "-I 99 -i 20,5" to make sure the results are not completely random. The result I got was... wait for it... 10359.48. Even with 1400 bytes of request/response size, I still get 3257.53. As the highest result in the Phoronix article is 918.92, I feel Mr. Larabel owes us some information about what he was actually testing and how.
I just hope the next article isn't going to present TCP_STREAM results (measured on gigabit ethernet).
Some tests show variance so high that the results should have been discarded and tests repeated [...] netperf has "-I" and "-i" parameters to make things easier. He runs the same test with two different lengths [...] the results should be the same within a margin of statistic error; if they are not, it should be a warning sign that something was wrong. [...]
One thing that really stands out is the UDP_RR test on Tumbleweed, in particular the 360s one. The variance itself [tells] the test went completely bonkers [...] even upper end of the indicated interval is still way below the results of the 60s test. [...] I quickly ran netperf UDP_RR between my two machines, one running 42.3 with 4.15.8 Kernel:stable kernel (i.e. essentially Tumbleweed), the other Tumbleweed with 4.16-rc4 kernel from Kernel:HEAD. The hardware is definitely worse than Mr. Larabel's and NICs aren't anything special either (on-board Realtek 8168evl and common consumer grade Intel (82541GI)). Both machines are running a KDE desktop (I only wanted to get some idea about the results) so I used "-I 99 -i 20,5" to make sure the results are not completely random. The result I got was... wait for it... 10359.48. Even with 1400 bytes of request/response size, I still get 3257.53. As the highest result in the Phoronix article is 918.92, I feel Mr. Larabel owes us some information about what he was actually testing and how.
I just hope the next article isn't going to present TCP_STREAM results (measured on gigabit ethernet).
Comment