Bufferbloat To Be Fought In Linux 3.3 With BQL

[phoronix]

Phoronix: Bufferbloat To Be Fought In Linux 3.3 With BQL

Another feature coming to the Linux 3.3 kernel is Byte Queue Limits (BQL), which attempts to fight "bufferbloat" in networking...

http://www.phoronix.com/vr.php?view=MTAzODg

[elanthis]

If any Phoronix readers do do ...

Sometimes the immature little kid in me just can't resist pointing out that phrase.

[GreatEmerald]

Sounds good, but what does that improve, specifically? Network latency? On what hardware, or is it not hardware specific?

Thinking of which, are there any network tests in PTS? I could only find one mentioned in OpenBenchmarking, and it's a loopback one, so it's rather synthetic. Either way it would be nice to see how Linux compares to Windows on the network speed front, too, given that optimisations are actually possible here.

[Sidicas]

Sounds good, but what does that improve, specifically? Network latency? On what hardware, or is it not hardware specific?

This is related to packet shaping..
If you've got one PC on your network that requires low latency and another PC on your network that's uploading videos to youtube / running bittorrent / etc.. Then the other PC can plug up your Internet connection, regardless of how fast it is and it can cause your first hop pings to go through the roof to the point where the network game isn't playable...

So that's where packet shaping comes in... Where you have round-robin types of packet shapers that round-robin between network connections or source IP address to attempt to give everybody their fair share of bandwidth (or not, if you configure it otherwise).. I believe the current implementation pretty much treats all packets the same, regardless of their size when they get loaded into the buffers/queues... So if one PC is making hundreds of tiny packets and another PC is making hundreds of full size packets, then the PC making the larger size packets will get more bandwidth than it should because it currently counts packets, not bytes.

I'm surprised that they have this, considering that i've heard in the past that byte counting like this causes a lot of overhead.. I guess they found a creative way to resolve that problem.

I use packet shaping a lot, because I have a lot of family who upload to Youtube and I've also got folding@home which uploads large work units to stanford's servers.. Sometimes there is so much data to be sent, that without my packet shaping, the ACK packets end up getting queued at the end of a massive FIFO queue of large data packets to YouTube / stanfords servers / etc.. Which causes web browsing to be very slow, even though it's the upstream bandwidth that's the problem.
That's why I use packet shaping as it makes sure those ACK packets get loaded into their own queue, and also get priority, so that browsing the web is always speedy even when there's a dozen HD uploads going simultaneously on a 3Mbit / 768 Kbit connection.

In short, a good packet shaper can prevent any PC on your network from hogging your network connection and can almost guarantee a constant low ping provided to PCs that aren't using more than their fair share of bandwidth... Regardless of how fast or slow your Internet connection is. It's incredibly useful.

This improvement, makes the packet shapers more fair / balanced than they have ever been in the past, which is very nice. Even the expensive gaming routers don't do byte counting like this so having a linux box as a router certainly has it's perks.

[89c51]

Getting this to work end to end means that every piece of HW in the chain must have anti bufferbloat code in it right???

Many modem/rooters run on linux and at some point they ll use the kernel but what about the HW run by ISP etc. ?

[Sidicas]

Getting this to work end to end means that every piece of HW in the chain must have anti bufferbloat code in it right???

No.. If you artificially throttle the bandwidth so that you don't buffer on a device that's upstream (towards ISP) of your linux box then it's not a problem on that device.. For example, my DSL router / modem has got very large FIFO buffers in it, but since my Linux router feeds it data at the same speed of the Internet connection, the buffers on that device remain nearly empty, all the time. If I take my Linux router out of the network, then PCs on the network will plug up my DSL router's large buffers with data and the ping to the Internet will go through the roof (from 35ms to over 1000ms)

When you do packet shaping, you want to make sure that all the buffering happens on the device that's doing the shaping.. Everything that's both downstream and upstream needs to have no queues of packets or it won't work.

It's not a bad idea to have anti-bufferbloat on all devices, but it's also not necessary on any more than 1 device if you're very accurate with your packet shaping and your available internet bandwidth remains stable.

[devius]

If you've got one PC on your network that requires low latency and another PC on your network that's uploading videos to youtube / running bittorrent / etc.. Then the other PC can plug up your Internet connection, regardless of how fast it is

I can relate to that. Tell me more...

I have a lot of family who upload to Youtube and I've also got folding@home which uploads large work units to stanford's servers.. Sometimes there is so much data to be sent, that without my packet shaping, the ACK packets end up getting queued at the end of a massive FIFO queue of large data packets to YouTube / stanfords servers / etc.. Which causes web browsing to be very slow, even though it's the upstream bandwidth that's the problem.

I always wondered why would the uploads be choking the downloads. That makes sense. What can I do about it?

That's why I use packet shaping as it makes sure those ACK packets get loaded into their own queue, and also get priority, so that browsing the web is always speedy even when there's a dozen HD uploads going simultaneously on a 3Mbit / 768 Kbit connection.

Sold!! Where can I get one of those "packet shapers" Do I need to have a custom linux device acting as router? Or can I change something in my PCs to make it work?

[dacresbu]

traffic shaping?

http://bit.ly/yfsgFl this link might help

I can relate to that. Tell me more...



I always wondered why would the uploads be choking the downloads. That makes sense. What can I do about it?



Sold!! Where can I get one of those "packet shapers" Do I need to have a custom linux device acting as router? Or can I change something in my PCs to make it work?

[Sidicas]

I always wondered why would the uploads be choking the downloads. That makes sense. What can I do about it?

Yea, it's actually really quite simple.. In TCP/IP, if the ACK / acknowledgement packets are delayed then it's assumes that there is a bottleneck and the server sending data to you will slow down the speed that it's sending data to you.. So you need to make sure that when your PCs receive data, they're able to get ACK packets back to the server sending the data to you as quick as possible or that server sending the data will slow down the speed that it's sending data to you while it waits for your ACK packets. For this reason, it's a good idea to give those tiny outgoing ACK packets priority over large outgoing data packets (uploads to youTube, etc.) to make sure they go through ASAP to prevent your download speeds from falling off a cliff.

So instead of having this long queue / buffer of a messy mix of ACK and data packets, the packet shaper takes all ACK packets and puts them to the front of the queue, even though the data packets might have been there first.. ACK packets are very tiny so they pretty much go through instantly.

Sold!! Where can I get one of those "packet shapers" Do I need to have a custom linux device acting as router? Or can I change something in my PCs to make it work?

Well, you need to set up a Linux PC between your internet gateway and the rest of your home network.. You can do it with 2 ethernet ports and configuring a software bridge with packet forwarding between the 2 ethernet ports. As far as setting up the packet shaping, there are a bunch of HOWTOs out there...

There's also a generic packet shaper called "wondershaper" which is available for almost all distros. apt-get install wondershaper
Or you can read all the HOWTOs and build your own configuration script.

[dtaht]

most of this thread was pretty good but...

"So instead of having this long queue / buffer of a messy mix of ACK and data packets, the packet shaper takes all ACK packets and puts them to the front of the queue, even though the data packets might have been there first.. ACK packets are very tiny so they pretty much go through instantly."

What you want is more 'fairness' between flows, and possibly a little boost to small acks, but not a lot. That's what packet shapers do, they generally do not look specifically at ACKs. So if you have 20 things going on, each of the 20 streams gets a proportional amount of bandwidth.

the wondershaper is a little different in that it does also in addition, try to give acks priority.

The wondershaper as shipped by many vendors is flawed (no ipv6 support, broken ecn support). There are other shapers out there.