Microsoft & Others Form The eBPF Foundation

[Jakobson]

Okay. Can you change kernel config flag at boot time and/or runtime?

At runtime via procfs if not set on by default:
https://elixir.bootlin.com/linux/lat...pf/Kconfig#L38

[Alex/AT]

At runtime via procfs if not set on by default:
https://elixir.bootlin.com/linux/lat...pf/Kconfig#L38

I don't see anything like 'disable bpf' under /proc there, do you?

There are few JIT options and one 'disable unprivileged BPF' option. Yes, I assume it can theoretically be okay with the second, but it still leaves some loopholes. For now rebuilding distro kernels becomes the only option to get rid of it on any critical and semi-critical instances, and this is ultimately bad.

[ryao]

Ten years from now when security researchers find all the security holes, there are going to be some painful mitigations for dealing with this.

It is a feature meant for debugging. It is a security hole by definition when accessible to untrusted people, although that does not mean that you cannot design secure systems using it. You can design secure systems that run with kgdb attached too. You just need to keep the interface away from untrusted people. eBPF typically requires root for this reason.

[ryao]

I don't see anything like 'disable bpf' under /proc there, do you?

There are few JIT options and one 'disable unprivileged BPF' option. Yes, I assume it can theoretically be okay with the second, but it still leaves some loopholes. For now rebuilding distro kernels becomes the only option to get rid of it on any critical and semi-critical instances, and this is ultimately bad.

You can build the kernel without the syscall. I would not advise it unless you are sure that you are never going to be debugging your machine. If you do not have debug symbols though, it is semi-useless, so building your kernel without it is not so bad. Just don’t expect to be able to file useful bug reports for hard to reproduce on developer system kernel issues as developers will not be able to ask you to run eBPF programs to get data on the issue.

[Alex/AT]

It is a feature meant for debugging.

Isn't it logical feature meant for debugging has nothing to do with the general network stack (except debugging), and should be enabled in debug kernel builds only.

You can build the kernel without the syscall.

With gentoo it's one thing, but you probably understand rebuilding kernel in mainline binary distros means you're off support for it, and you also need to do it with every update, so the bugfix cycle prolongs with that. So to disable a potential security issue, we effectively need to workaround other security issues. This doesn't stand as anything by me but deliberately introduced injection point you can't easily get rid of.

[ryao]

Isn't it logical feature meant for debugging has nothing to do with the general network stack (except debugging), and should be enabled in debug kernel builds only.

It was originally the Berkeley Packet Filter. It was extended, which made eBPF. Outside of networking, it is used for debugging. As for enabling it in debug builds only, when an end user reports a problem that only occurs on the user’s PC and the developer needs information from that PC, having eBPF present so that the user can run some commands to get data regarding the problem is helpful.

Without debug symbols, its ability to be used is limited, but most distributions have kernel symbol packages. They build the kernels as “debug” kernels and strip the symbols into debug symbol packages to produce the binaries that they distribute. Disabling it at the build system level would not make sense for the people who install kernel debug symbols.

This doesn't stand as anything by me but deliberately introduced injection point you can't easily get rid of.

This was introduced several years ago. It is typically restricted to root and once an attacker gets root (unless it is inside a container), it is game over. I don’t see this hurting security.

I know some guys writing an eBPF auditd replacement that is far more performant than auditd. At present, auditd tends to cause systems to slow to a crawl. Their WIP eBPF based prototype does not. If they succeed, eBPF could one day be a feature that improves security by making auditing something you could reasonably do on production machines outside government environments.

Anyway, eBPF is a great feature that has plenty of potential. Want to know the VFS cache hit rate? Use eBPF. Want to know which files are frequently accessed? Use eBPF. Want to know the distribution of filesystem/syscall/network latencies? Use eBPF. The iovisor/bcc project on GitHub makes doing such things with eBPF easy for system administrators. The sheer number of things it enables is far more than we would have with hard coded instrumentation in the kernel that only serves to bloat the kernel when not in use. eBPF gives us basically zero additional kernel bloat when not in use.

It also allows for debugging high frequency events, which was impossible with other means. ftrace for example imposed too much overhead to be used for frequently occurring events that lasted hundreds of nanoseconds because the overhead would cause the data to not match what happens on a production system. eBPF is low overhead enough that you can use it for that. It has been used inside Netflix for that purpose for years. See Brendan Gregg’s presentations for examples.