Announcement

Collapse
No announcement yet.

Linux Distributions vs. BSDs With netperf & iperf3 Network Performance

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • teknoraver
    replied
    Sorry for resume an old thread, but this test is wrong on so many levels.
    • You may wonder why in so many tests, the results are the same for every OS: this is because you're reaching the line rate.
    • The network cards have multiple hardware queues, and put the received packets in a queue depending on the L3/L4 hash. Every hardware queue is handled by a kernel thread, so if the traffic comes from few connections, you may use only a few cores. This is a standard technology named RSS
    • Software like iperf3 and netperf can handle only the fraction of the traffic of a real traffic generator, they are tiny toys compared to, eg. Anritsu or Xena
    • In the networking industry, the measure unit is always the packet per second (and its multiplies) , not bit per second. This because handling a packet needs the same effort if it's a 64 byte or a 9000 byte jumbo packet. If you get the datasheet of any router, NIC or software, the specs always says mpps (millions of packets per second), and not gbit/s
    If you don't want, or you can't get an HW traffic generator, you may get some much better result with iperf by limiting the UDP packet size or the card MTU to something really small, like 64 byte, and post the results in mpps instead of mbit. And be sure to use a very high connection count, at least 4x the number of CPUs, if you want that the hardware queues are evenly filled and all the CPUs are working.

    If you are just curious, this is an IP forwarding test made with 5000 connections and a 10 Gbit card, with a real generator


    Leave a comment:


  • nasyt
    replied
    Originally posted by Pawlerson View Post
    Two questions:

    1. Was firewall enabled in benchmarked systems? (in Linux distributions it's usually enabled, but it's probably not enabled by default in FreeBSD).
    2. Does it matter?
    For the second question: Yes it does matter, as you and your retinue are always pouring so much hatred over ***BSD and, to a larger extend, Solaris. When it comes to other OSes (BSD, Solaris excluded) Linux is just one OS among others.

    Leave a comment:


  • F.Ultra
    replied
    Originally posted by indepe View Post

    Well both are SOCK_DGRAM so I wasn't distinguishing UDP and ICMP.....however that sounds like Ping might have quite some overhead compared to repeated single byte UDP and TCP exchanges.

    So 0.4 ms might also correspond to 0.09 / 0.4 * 19,200 /sec = 4,320 /sec. Even better but still far from 19,200 /sec.

    What's your network card, if I may ask? Something like Intel I350?
    It's a i350 chip yes (SuperMicro X10RW-E motherboard with built in NICs). Could be that the Linux kernel is prioritizing tcp/udp packets over icmp since good performance is way more important there than for "ping", also since the TCP_RR test is done on an established connection the data stream is going through less hoops in the kernel than a random icmp ping would.

    Leave a comment:


  • indepe
    replied
    Originally posted by F.Ultra View Post

    For normal ping I have at the moment:
    Code:
    --- lon.x.com ping statistics ---
    62 packets transmitted, 62 received, 0% packet loss, time 61000ms
    rtt min/avg/max/mdev = 0.039/0.090/0.128/0.021 ms
    This is icmp pings however and not udp or tcp ones and the switch used is a HP 2910al-24G.
    Well both are SOCK_DGRAM so I wasn't distinguishing UDP and ICMP.....however that sounds like Ping might have quite some overhead compared to repeated single byte UDP and TCP exchanges.

    So 0.4 ms might also correspond to 0.09 / 0.4 * 19,200 /sec = 4,320 /sec. Even better but still far from 19,200 /sec.

    What's your network card, if I may ask? Something like Intel I350?

    Leave a comment:


  • F.Ultra
    replied
    Originally posted by indepe View Post
    Unexpectedly, one of the computers in my local network responds to ping requests (UDP). The time I get with Fedora 25 is 0.4 ms, which I guess corresponds to a transaction rate of roughly 2,500 / sec, somewhere midway between your 19,000 / sec and Michael's ~150 / sec on F25. These differences are enormous.

    (EDIT: Fedora 25 Workstation, that is.)
    For normal ping I have at the moment:
    Code:
    --- lon.x.com ping statistics ---
    62 packets transmitted, 62 received, 0% packet loss, time 61000ms
    rtt min/avg/max/mdev = 0.039/0.090/0.128/0.021 ms
    This is icmp pings however and not udp or tcp ones and the switch used is a HP 2910al-24G.

    Leave a comment:


  • aht0
    replied
    Originally posted by F.Ultra View Post
    There must definitely be something strange going on between Linux and the particular NIC used in the tested machine.
    Would not be first or even tenth time when there has been some regression in Intel's NIC Linux driver.. Just googling "intel nic regression linux" returns near 200k results.

    Leave a comment:


  • indepe
    replied
    Unexpectedly, one of the computers in my local network responds to ping requests (UDP). The time I get with Fedora 25 is 0.4 ms, which I guess corresponds to a transaction rate of roughly 2,500 / sec, somewhere midway between your 19,000 / sec and Michael's ~150 / sec on F25. These differences are enormous.

    (EDIT: Fedora 25 Workstation, that is.)
    Last edited by indepe; 16 December 2016, 04:43 PM.

    Leave a comment:


  • indepe
    replied
    Transaction rate > 19,000 per sec? That is so much better than both previous results that one has to ask: what is going on there?

    Leave a comment:


  • F.Ultra
    replied
    There must definitely be something strange going on between Linux and the particular NIC used in the tested machine. Just tested myself on a much leaner CPU (Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz) using the Intel igb NIC driver and had no problem to saturate 1Gbps (and this was tested on working servers as well) and I did also use the botch standard kernel (3.13.0-101-generic) in Ubuntu 12.04LTS so no low-latency kernel:

    Code:
    [email protected]:~# netperf -H x.159.94.204 -t TCP_STREAM
    MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to x.159.94.204 () port 0 AF_INET : demo
    Recv   Send    Send                          
    Socket Socket  Message  Elapsed              
    Size   Size    Size     Time     Throughput  
    bytes  bytes   bytes    secs.    10^6bits/sec  
    
     87380  16384  16384    10.02     935.49  
    
    [email protected]:~# netperf -H x.159.94.204 -t TCP_MAERTS
    MIGRATED TCP MAERTS TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to x.159.94.204 () port 0 AF_INET : demo
    Recv   Send    Send                          
    Socket Socket  Message  Elapsed              
    Size   Size    Size     Time     Throughput  
    bytes  bytes   bytes    secs.    10^6bits/sec  
    
     87380  16384  16384    10.00     937.93  
    
    [email protected]:~# netperf -H x.159.94.204 -t TCP_RR
    MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to x.159.94.204 () port 0 AF_INET : demo : first burst 0
    Local /Remote
    Socket Size   Request  Resp.   Elapsed  Trans.
    Send   Recv   Size     Size    Time     Rate        
    bytes  Bytes  bytes    bytes   secs.    per sec  
    
    16384  87380  1        1       10.00    19201.19  
    16384  87380
    
    [email protected]:~# netperf -H x.159.94.204 -t UDP_RR
    MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to x.159.94.204 () port 0 AF_INET : demo : first burst 0
    Local /Remote
    Socket Size   Request  Resp.   Elapsed  Trans.
    Send   Recv   Size     Size    Time     Rate        
    bytes  Bytes  bytes    bytes   secs.    per sec  
    
    212992 212992 1        1       10.00    19248.45  
    212992 212992

    Leave a comment:


  • aht0
    replied
    Originally posted by starshipeleven View Post
    Main reason Pulseaudio isn't used outside Linux is that some other OS have already their own sound system and it is fine (Windows or macOS), and *BSDs have relatively better situation with audio than Linux for basic stuff, and since their notion of PC or server is still firmly rooted in the early 90's they don't need all fancy things that Pulse allows, like for example 5.1 audio setups or bluetooth audio or whatever.
    I will forego the systemd portion of the debate now, you know I dislike it anyway.

    "Pulseaudio" exists outside Linux. You can install it on FreeBSD for example if you feel like masochist. It's older version though. And AFAIK in the end it's going to still use OSS backend.

    You are wrong partially. A lot of things Pulseaudio was meant to do, were doable in BSDs already on kernel level so it's in large parts superfluous to existing functionality.
    Last edited by aht0; 12 December 2016, 05:32 PM. Reason: typod fixed (I hope)

    Leave a comment:

Working...
X