Linux 6.12 NFS Adds LOCALIO Protocol For "Extreme" Performance Boost

[phoronix]

Phoronix: Linux 6.12 NFS Adds LOCALIO Protocol For "Extreme" Performance Boost

The Network File System (NFS) changes have been merged for the ongoing Linux 6.12 development cycle. Notable this time with NFS is adding LOCALIO protocol extension support that can lead to fairly "extreme" performance improvements in scenarios where the NFS client and server are on the same host...

https://www.phoronix.com/news/Linux-6.12-NFS

[Chugworth]

I'm surprised people still take NFS seriously when the only form of security it provides is checking the source IP address, and it transmits the data unencrypted over the network. Yes, I know there is a way to add data encryption. I've looked into it and it seems to be a convoluted mess to implement with very little good documentation.

[intelfx]

I'm surprised people still take NFS seriously when the only form of security it provides is checking the source IP address, and it transmits the data unencrypted over the network.

Tell me you have no idea without telling you have no idea.

Yes, I know there is a way to add data encryption. I've looked into it and it seems to be a convoluted mess to implement with very little good documentation.

Right. If we ignore authentication and encryption, then NFS has no authentication and encryption.

[NateHubbard]

in scenarios where the NFS client and server are on the same host.

That doesn't strike me as something I'm going to be able to make much use of. What I'm currently exporting is already directly accessible on the server.
But, yeah, I get it. Seeing that "extreme performance" headline got my attention though.

[EphemeralEft]

That doesn't strike me as something I'm going to be able to make much use of. What I'm currently exporting is already directly accessible on the server.

I wonder if there’s any scenario where LOCALIO would work but bind-mounting directories wouldn’t. I also wonder what kind of overhead this will have for non-LOCALIO scenarios.

[dragorth]

I could see this as being useful on a Proxmox host with one Open Media Vault container and then other containers that use that as the backing storage, such as NextCloud. Cool.

[pgoetz]

I'm surprised people still take NFS seriously when the only form of security it provides is checking the source IP address, and it transmits the data unencrypted over the network. Yes, I know there is a way to add data encryption. I've looked into it and it seems to be a convoluted mess to implement with very little good documentation.

While I completely agree with about the lack of good documentation, the secure version of NFS is designed to work with Kerberos and isn't particularly convoluted. Not only is NFS widely used, particularly in the HPC/academic world, but the software is under very active development, as evidenced by the scores of patches posted to the NFS developers list.

[Chugworth]

Tell me you have no idea without telling you have no idea.



Right. If we ignore authentication and encryption, then NFS has no authentication and encryption.

Well first of all it's not a complete package. You need add-ons from Oracle to make it work. And the very limited information about the encryption add-on leads me to believe that few people are actually using it. Hence the surprise that NFS is still a thing in today's highly security-focused environment.

Might as well bring Telnet back.

[trapexit]

I wonder if there’s any scenario where LOCALIO would work but bind-mounting directories wouldn’t. I also wonder what kind of overhead this will have for non-LOCALIO scenarios.

You should not be using the underlying filesystem directly when exported via NFS. For some reason this isn't widely mentioned because perhaps it's not a common need for people but as the author of mergerfs I see folks do this *all the time* and don't think twice. With SMB, mergerfs, NFS, overlayfs, etc. mergerfs at least can be configured so caching is largely disabled so there is little risk in normal usage but if you have some sort of layered filesystem setup you should *not* be accessing the underlying filesystem out of band.

In this case if you have containers that could live anywhere you don't really want to conditionally manage the storage. NFS share lives on host X... doesn't matter where the container lives it mounts from X. No "if local bindmount, else nfs"

[elvis]

I'm surprised people still take NFS seriously when the only form of security it provides is checking the source IP address, and it transmits the data unencrypted over the network. Yes, I know there is a way to add data encryption. I've looked into it and it seems to be a convoluted mess to implement with very little good documentation.

HPC/VFX systems architect here. Yup, we know it's "insecure". But NFSv3 is still far, far more performant than other protocols.

SMB3 has nice stuff in it, but really requires a lot of effort and annoying network design to utilise multi channel and get performance up to the levels required. SMB over QUIC is very exciting, but that's going to take a while before we see it arrive for easy consumption in ksmbd and on clients. SMB over RDMA is just painful, and breaks constantly.

NFSv4 is nice and secure, but the performance overheads are annoying. If you're pushing 100GbE+ workloads, that matters. Likewise pNFS has its upsides, but clustered storage is always tricky for things that need good IO performance, versus just ensuring you don't saturate a single point in your cluster. For smaller (single digit petabyte) workloads, it's quite easy just to fill a device full of NVME drives and export that over traditional NFSv3 to overcome those bottlenecks.

Same "clustering is annoying" comments apply to things like GPFS, Lustre, etc. Yes these are marvellous at very large scale where you want a single namespace. But people like Weta still use lots and lots of individual NFSv3 NAS devices and a globally managed autofs via Puppet/Ansible/Salt type tools, simple because it's cheaper, easier and more performant for their industry.

S3 (the protocol, not the AWS service) and object storage in general are great, if your applications can use them. We're trying very hard to migrate people over to object storage, but HPC and VFX industries still have a huge volume of tools and applications that demand "old fashioned" POSIX style file systems. We try to push users and vendors alike, but that's a long road, and we're only at the start.

Would I expose NFSv3 to an untrusted network? Hell no. Would I roll out NFSv3 in 2024 on a controlled, locked down, internal, storage-dedicated network with thousands of private compute nodes smashing my storage? Yup, you bet. Am doing just that right now. Coupled with 100GbE+ and jumbo frames, NFSv3 still beats the pants off anything else.

Like literally anything in computing, there is no "one size fits all" solution. If you think someone using something is surprising, go and ask the people using it why they do. You'll probably learn about a tonne of industry-specific requirements you never even knew existed. This is business diversity in action.