Linux 5.6 To Bring FQ-PIE Packet Scheduler To Help Fight Bufferbloat

[phoronix]

Phoronix: Linux 5.6 To Bring FQ-PIE Packet Scheduler To Help Fight Bufferbloat

In addition to WireGuard being part of "net-next" as the networking subsystem material targeting the upcoming Linux 5.6 cycle, there is another big last minute addition to the networking space: the Flow Queue PIE packet scheduler has been merged...

http://www.phoronix.com/scan.php?page=news_item&px=Linux-5.6-FQ-PIE-Packet-Sched

[c117152]

How suitable it is to home gateways? The comparisons over at https://fq-pie.herokuapp.com/ suggests it's better than fq-codel and pie in both fairness and performance... But it says nothing about the RAM requirement aside from the 32MB default for total queues mentioned in the commit so how does it really compare to cake under real world conditions?

[Raka555]

I had pretty amazing results with cake.
I prefer it over fq_codel and it is now part of main line since 4.19. I always had to compile it out of tree before.

https://www.bufferbloat.net/projects/codel/wiki/Cake/

It has been running reliably for years on an Orange pi zero with 256MB RAM.

We have ADSL with 1Mb/s up and 10Mb/s down link speed at the place where I work.

In the past the internet would just grind to a halt when the Apple devices started cloud syncing.
They would totally swamp the outgoing bandwidth.
Now nobody notices when an Apple device is put on charge

[boxie]

Michael Could we have a congestion control comparison as one of your next diligent investigations? (please )

How would you go about testing that? do you want everyone with the same latency? or with bursty access to bandwidth when you have some hogs? genuinely curious

[Mangix]

How suitable it is to home gateways? The comparisons over at https://fq-pie.herokuapp.com/ suggests it's better than fq-codel and pie in both fairness and performance... But it says nothing about the RAM requirement aside from the 32MB default for total queues mentioned in the commit so how does it really compare to cake under real world conditions?

Probably better than fq-codel.

The point of PIE vs CoDel is lower RAM usage. In OpenWrt, fq-codel is patched to use less RAM. This might be a better fit there.

[starshipeleven]

How suitable it is to home gateways? The comparisons over at https://fq-pie.herokuapp.com/ suggests it's better than fq-codel and pie in both fairness and performance... But it says nothing about the RAM requirement aside from the 32MB default for total queues mentioned in the commit so how does it really compare to cake under real world conditions?

it is worth mentioning that devices with 32MB of RAM are on their last legs in projects like OpenWrt, and that now the bare minimum for normal operation is 64MB, with most new decent devices have 128-256MB, and high-end ones having 512MB.

THat said, anyone wanting to NAT more than 200MBit/s is usually recommended to buy/use x86 hardware anyway.

[c117152]

fq-codel is patched to use less RAM. This might be a better fit there.

Good to know. Hopefully they'll be able to get FQ-PIE working on low RAM too.

it is worth mentioning that devices with 32MB of RAM are on their last legs in projects like OpenWrt, and that now the bare minimum for normal operation is 64MB, with most new decent devices have 128-256MB, and high-end ones having 512MB.

THat said, anyone wanting to NAT more than 200MBit/s is usually recommended to buy/use x86 hardware anyway.

That's just the WPA3 and packet inspection requirements from what I understood. I think it will be possible to run OpenWrt with those turned off just fine so long as you have 8-16MB flash and know your way around the image builder. Well, I hope...

[Mangix]

The OpenWrt issue with 4MB has to do with using newer kernels. For those devices, DD-WRT is a better bet (still on kernel 3.10).

[dtaht]

A couple notes.

Pie is the default AQM in DOCSIS 3.1. It was kind of my hope that fq-pie would be easier to implement in some hardware that has proven hard to fq-codel.

Notably pie drops at the tail of the queue and is O(1) on egress from the queue, where codel drops at the head and is not O(1), so it's mildly harder (in a hardware implementation) to do codel "right" without an additional algorithm underneath it (bql in linux) to supply a very small amount of additional buffering. (on the other hand codel's drop from head does signal back faster even with that small amount of additional buffering).

competition in AQMs is GOOD. It's not the "point of pie vs codel to lower ram usage", having algorithmic diversity and competitetive research is the point. Ram usage looks to be about the same with both algorithms.

As for the 32MB default memory limit - that's an artifact of making fq_codel "just work" without tuning in the DC. The openwrt folk are right to reduce the default for (not just fq-codel but pie and cake) in their most extremely memory limited devices.

As to which one is better? Well, in part, I supported the further development of sch_cake pie and fq-pie and the inclusion into the linux kernel so independent benchmarkers to have a go at which is best on their workloads. Any benchmark I were to do would be considered biased! I'd like it if far more folk developed good tests for their workloads - and tried using these algorithms day in and day out to gauge user experience more abstractly. Turn on fq-pie one week, see how it went, cake another, etc.

One observation from looking at https://fq-pie.herokuapp.com/ is that BBRv1 overwhelms conventional cubic traffic because of its lack of response to the gentle packet loss a single queue AQM induces, where the fq versions are far more even, and BBRv1 also thrusts too hard against against fq_codel's lower RTT target which I kind of hope gets resolved in BBRv2.

Another observation is from the tcp_12up test where there is a periodic glitch in the fq-pie algo. Perhaps this is a bug?

I would really like all testers of this stuff to use the flent rrul test in addition to simple up or down tests. Mixed traffic (with acks) really shows the flaws in a single queue AQM in particular. Normal users don't just sit there and download stuff, and then switch to uploading stuff...

Anyway... Cake has a more advanced version of the codel algorithm (called cobalt), some features of which are under consideration to fold back into fq_codel. Cake is IMHO, the gold standard at this point for a shaper + fq + AQM - but I'd really like to see benchmarks of all the algorithms at a gbit and above at line rate.

Progress continues, but I think any of these algorithms are worth deploying just about everywhere you can. The results are palpable.

If you would like some intuition, and a few laughs, see my recent talk at:

https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/

[starshipeleven]

That's just the WPA3 and packet inspection requirements from what I understood.

No, devices with 32MB of RAM were barely usable even when the project was split and called LEDE for a release (17.something). I have a bunch of them that have less than 1MB of RAM usable at all times.

It's as simple as kernel and applications needing more RAM as time goes on.

And I'm not talking of Qualcomm wifi being optimized like complete shit and wasting 100-150MB per wifi radio because of buffers (this was recently worked around in OpenWrt by reducing the buffer sizes for devices that don't have a ton of RAM).