Netfilter Releases Nftables 1.0

[oibaf]

I switched to iptables a long time ago, so I don't really follow who reports where. Since iptables is working fine, there is nothing to report

iptables in firewalld is deprecated, will be removed in the future:
See https://firewalld.org/2021/07/firewalld-1-0-0-release

[silentcreek]

I don't use nftables because it has terrible problems with ipsets, for example a 1MB list of addresses consumes 1GB of memory. Apparently something to do with how nftables manages addresses in memory. Which makes nftables impossible to use since I've got a 5MB ipset of blocked addresses.

While I have experienced (and most of which reported) a number of issues with nftables sets (especially) large ones, I cannot confirm this one. Maybe this one only occurs in conjunction with firewalld?

I use large sets or lists of ip addresses and networks as well on some of my systems. I have a small system (used as a VPN server) with just 1GB of ram here and the largest list of networks that I currently use to populate a nftables set is 2 megabytes of uncompressed text or roughly 125000 networks (and this is just one of several sets I use). My total memory usage on this little machine is currently ~200 megabytes for the entire system(!) and including buffers and caches (Debian 11 with systemd and a few background services such as an OpenSSH, OpenVPN and a few more). I also didn't see any meaningful change in memory usage when I switched from iptables to nftables.

The only issue (of those I experienced) that is not resolved by now is that atomically replacing or updating such large sets still fails (and I have yet to report this one because I just upgraded to Debian 11 and tested this again). I have to flush the set first before I can repopulate it. With the smaller sets this works fine now (didn't on older nftables releases and kernels), but for this particular one it doesn't.

So, yes, from my experience, nftables sets aren't as mature and robust as ipsets. But aside from that, I very much prefer nftables over iptables. It's just so much easire to read and write.

[oibaf]

BTW, I found the issue is being discussed here:

• https://github.com/firewalld/firewalld/issues/738
• https://bugzilla.redhat.com/show_bug.cgi?id=1908127

[silentcreek]

BTW, I found the issue is being discussed here:

• https://github.com/firewalld/firewalld/issues/738
• https://bugzilla.redhat.com/show_bug.cgi?id=1908127

Thanks, this is interesting. I could reproduce the second issue. I took the example set from RedHat's bugzilla report and tried to add it to my ruleset on said little machine with just 1 GB of RAM. It failed. It consumed all memory and swapped out another gigabyte until nft eventually returned "Killed". The reason I didn't see this before is that this example list is not just 1MB large but more than 17MB with over 1.1 million entries, so almost 10 times larger than my largest set.

After this, I also looked at the memory consumption when I reload my sets (again, my largest list being ~125000 networks). When nft reloads the lists, the total memory consumption on my system climbs to about 500MB and when it's done, memory consumption drops again to around 200MB as mentioned before. So, the reason I never saw this particular issue is that my lists are just not large enough. But this also shows that it's not sets per se that require a lot of memory, but the process of loading them, once nft is done, memory consumption goes back to normal instantaneously again. (Btw. atomic reloads also require much more memory than loading a set initially. The aforementioned ~500MB of memory consumption are for a reload operation. When Ioading the same ruleset and sets from scratch memory consumption is somewhere between 300-350MB.)

[sdack]

I don't use nftables because it has terrible problems with ipsets, for example a 1MB list of addresses consumes 1GB of memory. Apparently something to do with how nftables manages addresses in memory. Which makes nftables impossible to use since I've got a 5MB ipset of blocked addresses.

Your problem is not with nftables, but with your way of using it. One does not block with a flat list of a million addresses. It forces the code to either do a linear search over all addresses for every incoming packet, or the code needs to build up hashes and trees, which costs memory obviously. The list itself is an unmanageable amount of addresses for a human. By the time you have compiled your list will it be outdated and you need to add new addresses to it and remove old ones. So you might as well block all and start using netmasks and other means. What do you do with the rest if the Internet? Are they all unblocked or what?

[bash2bash]

I don't have a problem because I use iptables, which works fine and has minimal memory usage.

About my use of blocked addresses, it is pretty simple, I block all of Asia plus some extras (Russia, Brazil, etc). This simply reduces my spam rate down to... zero... and blocks all hacking attempts, malware, wordpress scans, bots and other typical garbage that these countries produce. In addition, I make sure to block the entire OVH AS range. They are all permanently banned, never removed and will remain banned for the rest of the internet history.

btw, I use these IP ranges on all my servers, my client servers, friend servers, routers, mobile platforms, pretty much everything uses these and iptables has absolutely no problem with all that, runs fine and purrs like a kitten, unlike nftables that croaked and died.

PS:
I do have a bash script that automatically updates the country IP ranges and adds any new OVH ranges.

Your problem is not with nftables, but with your way of using it. One does not block with a flat list of a million addresses. It forces the code to either do a linear search over all addresses for every incoming packet, or the code needs to build up hashes and trees, which costs memory obviously. The list itself is an unmanageable amount of addresses for a human. By the time you have compiled your list will it be outdated and you need to add new addresses to it and remove old ones. So you might as well block all and start using netmasks and other means. What do you do with the rest if the Internet? Are they all unblocked or what?

[sdack]

... it is pretty simple, I block all of Asia plus some extras (Russia, Brazil, etc). ...

Yes, I thought you must be doing something pretty stupid. What you do is pointless from a technical point, because attackers regularly use IP spoofing to hide their origin. All you do with with your list is to flatter your ego, keep off those who do not hide their IP address and burden yourself with some list that you then update on every machine for no value.

You want to look into connection tracking for a start. It allows you to block all incoming traffic and to then only allow incoming traffic for which your computer has an active outgoing connection. So when you access i.e. a web server from your computer will your firewall automatically track the outgoing connection and not block the returning traffic from the web server. Once the connection closes does it then block also any excess traffic incoming from that web server. If you then still need to have ports open to allow for other incoming traffic, i.e. ssh, then you should again block all and only allow access from a few trusted machines, ideally none from the Internet but only the local network. There are then other methods like port-knocking and what not, which are all better than a flat list of a million addresses.

[bash2bash]

To the contrary, its quite simple and easy to block by country.

It seems like you don't know what you are talking about, which is rather sad.

I agree that a sophisticated attacker will use IP spoofing, a VPN or a proxy, to by-pass regional blocks, but those are just 1% of the attacks. The other 99% uses local IP addresses. Thus, blocking China and Russia, automatically blocks most attacks. Add the rest of Asia to that and you've managed to block that 99% I mention above. The 1% is more elusive and its blocked via other means.

I am not interested to put a full block and allow a few addresses because these are public facing servers (web, mail, etc) so that is not an option. Of course ssh is properly configured and protected to allow static IP addresses/locations.

Anyway, I suggest you take a look at worldwide malicious traffic, to get an idea about how things work, before you make silly comments.

Yes, I thought you must be doing something pretty stupid. What you do is pointless from a technical point, because attackers regularly use IP spoofing to hide their origin. All you do with with your list is to flatter your ego, keep off those who do not hide their IP address and burden yourself with some list that you then update on every machine for no value.

You want to look into connection tracking for a start. It allows you to block all incoming traffic and to then only allow incoming traffic for which your computer has an active outgoing connection. So when you access i.e. a web server from your computer will your firewall automatically track the outgoing connection and not block the returning traffic from the web server. Once the connection closes does it then block also any excess traffic incoming from that web server. If you then still need to have ports open to allow for other incoming traffic, i.e. ssh, then you should again block all and only allow access from a few trusted machines, ideally none from the Internet but only the local network. There are then other methods like port-knocking and what not, which are all better than a flat list of a million addresses.

[cueball]

Yes, I thought you must be doing something pretty stupid. What you do is pointless from a technical point, because attackers regularly use IP spoofing to hide their origin. All you do with with your list is to flatter your ego, keep off those who do not hide their IP address and burden yourself with some list that you then update on every machine for no value.

You want to look into connection tracking for a start. It allows you to block all incoming traffic and to then only allow incoming traffic for which your computer has an active outgoing connection. So when you access i.e. a web server from your computer will your firewall automatically track the outgoing connection and not block the returning traffic from the web server. Once the connection closes does it then block also any excess traffic incoming from that web server. If you then still need to have ports open to allow for other incoming traffic, i.e. ssh, then you should again block all and only allow access from a few trusted machines, ideally none from the Internet but only the local network. There are then other methods like port-knocking and what not, which are all better than a flat list of a million addresses.

nice view from your high horse?
I use geo blocking extensively. It makes sense for my use case, and obviously does also for the OP as they specifically said it was working for them.
OP also mentions they are using servers, which presumably is for, you know,... serving traffic..
so your entire paragraph around blocking all incoming traffic is probably pretty far off the mark.
such a judgy unhelpful post. lets at least try to help our community?

[sdack]

so your entire paragraph around blocking all incoming traffic is probably pretty far off the mark.

Probably?! You are just another idiot. Get smart when you want respect. And allow me to help! The source IP address in TCP/IP packets is not set in stone. It is not frozen, nor untouchable. Most malicious traffic has their source address spoofed, meaning, it has a source address that is not the actual source address. The bad guys do not want to get caught and make their traffic hard to trace on purpose. So they spoof the source address, do the same with the rest of their packets, and make them look like a believable accident, like a lonely packet ending up at the wrong address where it knocks innocently at one of your ports.

To think you could prevent attacks by using geography is as dumb as it gets and it is not more than racism. Even the average kid today already knows how to hide their address, while you still think using a flat list of a million addresses from Russia and China would do something. All it does is to slow your incoming traffic down, because every packet needs to get checked against each of these addresses. Give this a moment to think about, if you can... How much effort is it to scan the list? When you do it with a linear search then it saves you memory, but it is very slow. When you want it fast then you need to make trade-offs, use hashes and trees, which costs you memory.

So I am going to repeat it again. You both are idiots. All you actually do is to slow down or to stop valid traffic from getting to its destination, while malicious traffic has already outsmarted you. You have failed at solving your problem, and instead do you politicise and blame it on other countries. Trust me, I am helping you more than you currently can see.