Announcement

Collapse
No announcement yet.

Linux Work Culminating On A "READFILE" Syscall For Reading Small Files Efficiently

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • kreijack
    replied
    Originally posted by yoshi314 View Post
    i wonder what is the size limit on the file. otherwise weird things will happen.
    Originally posted by _Alex_ View Post

    I was actually wondering something similar: What happens if another process changes the size while the first process (system call) tries to read. Can it be used as an exploit? Something to that effect...
    In what it would be different from a read(2) syscall ?
    • The size of the data read has an upper bound to the "count" parameter of the syscall, which should be the size of the buffer of the syscall. Of course is legal that the read() can returns less data than request. The same applies to readfile()
    • Even the classic read could suffer for a changing of the file during the reading. So the program should handles this case too.

    Leave a comment:


  • coder
    replied
    Originally posted by ryao View Post
    That is not happening. Software will need to explicitly take advantage of this.
    I think the idea of having readfile() in libc is key for enabling software to use it. If it had been there all along, then this would just be a transparent, drop-in optimization.

    However, at least adding it now lets you write cross-platform userspace code with it. That's going to really help adoption.

    Leave a comment:


  • ryao
    replied
    Originally posted by d4ddi0 View Post
    If "small" is defined as under 4k, and it's handled in a backwards compatible way in libc (maybe make open() a noop, only actually "opening" if the file is large or opened for writing, and close() also potentially a noop), this could be an improvement for 90% of all file accesses.
    That is not happening. Software will need to explicitly take advantage of this.

    Leave a comment:


  • ryao
    replied
    Originally posted by jacob View Post
    Why does it need a new syscall? Couldn't an ioctl call do it?
    An ioctl would require opening the file first, which defeats the purpose of putting open, read and close into a single syscall.

    ioctls are also implemented in drivers rather than in the VFS, so ignoring the reality that it is not possible to do this via an ioctl (possibly via a device node in /dev) and somehow doing it anyway would be a mess. You really want an ioctl when controlling a device, not when doing a VFS operation.

    Leave a comment:


  • _Alex_
    replied
    Originally posted by yoshi314 View Post
    i wonder what is the size limit on the file. otherwise weird things will happen.
    I was actually wondering something similar: What happens if another process changes the size while the first process (system call) tries to read. Can it be used as an exploit? Something to that effect...

    Leave a comment:


  • JustinTurdeau
    replied
    Originally posted by jacob View Post
    Also there is an indirection in all cases: a new syscall is only a function from userland's prospective, at the kernel level it's nothing more than a number in a table.
    Right, but ioctls are essentially doing double dispatch. Once for the ioctl syscall itself and then again depending on the arguments. Why do double dispatch for no reason? Just to be contrarian?
    Last edited by JustinTurdeau; 25 May 2020, 04:24 PM.

    Leave a comment:


  • jacob
    replied
    Originally posted by pal666 View Post
    why? do you always add another switch case to same function with void pointer argument instead of introducing new function?
    this is really crazy and unfounded idea. it has no benefits, but it has real costs: you are losing type information by sending garbage arguments (...). ioctl was created to allow device drivers which can't introduce system calls to still be able to provide some driver-specific functionality, i.e. they differ per fd, while subj is global
    Not really. Type information is lost anyway at the kernel syscall interface (it doesn't know any type other than CPU register). Also there is an indirection in all cases: a new syscall is only a function from userland's prospective, at the kernel level it's nothing more than a number in a table.

    You are right that ioctl semantics depend on the particular fd but that's the whole point, to implement ad-hoc functionality that only makes sense for certain kernel objects. When using BTRFS for example file and directory fd's support all sorts of specific ioctls (clone, reflink etc.) The precise ioctl mechanism is a POSIX relic that I'm not particularly fond of, it could and should be replaced by some better interface management model, but the idea of having all objects referenced by fds and each fd offering methods that make sense for that particular object is IMHO sound and better than introducing more and more syscalls that only make sense in some cases and for some objects.

    Leave a comment:


  • yoshi314
    replied
    i wonder what is the size limit on the file. otherwise weird things will happen.

    Leave a comment:


  • akvadrako
    replied
    Originally posted by xpue View Post
    Perhaps also should consider making system calls to work with multiple files/whatever in one call, in batches.
    Sounds like something eBPF would be well suited for. You could install a little program like open();read();close();return and call it with one context switch.

    Leave a comment:


  • pal666
    replied
    Originally posted by jacob View Post
    No aversion, but they shouldn't be introduced at whim.
    why? do you always add another switch case to same function with void pointer argument instead of introducing new function?
    Originally posted by jacob View Post
    Ideally only really new concepts should have new syscalls, new operations on existing object types should be implemented using existing generic interfaces.
    this is really crazy and unfounded idea. it has no benefits, but it has real costs: you are losing type information by sending garbage arguments (...). ioctl was created to allow device drivers which can't introduce system calls to still be able to provide some driver-specific functionality, i.e. they differ per fd, while subj is global
    Last edited by pal666; 25 May 2020, 04:57 AM.

    Leave a comment:

Working...
X