Announcement

Collapse
No announcement yet.

Clang LTO Support Looks Like It Could Land For Linux 5.12

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • DanglingPointer
    replied
    dkms developers need to fix this!

    dkms needs to be able to dynamically figure out what build tools were used to build the kernel and use the same ones for the modules! Other's have had the same issue and raised a ticket against dkms. dkms was invented by Dell and github repo and issue tracker ticket for the bug is here...
    https://github.com/dell/dkms/issues/124

    If you are having the same problems with modules or like me with Virtualbox using dkms with a host compiled by llvm-clang; please add your voice and comment in the ticket above! Sooner this is fixed the sooner we can all benefit from a more optimised linux kernel for hosts running VMs.
    Last edited by DanglingPointer; 12 July 2021, 11:42 PM.

    Leave a comment:


  • DanglingPointer
    replied
    Originally posted by reba View Post

    Hi there. I just re-played the stuff in my posting and made some adjustments.
    For example bison was missing as a dependency and I added some snippets and explainations.
    And, yes, you're supposed to edit the Makefile of the linux headers.

    The module zramswap also breaks *sigh* so there might be others as well that are in need for a recompile.
    Maybe, after all, I am doing something wrong?
    Code:
    Jul 10 13:26:33 debian root[12072]: Starting Zram
    Jul 10 13:26:33 debian zramswap[12072]: <13>Jul 10 13:26:33 root: Starting Zram
    Jul 10 13:26:33 debian zramswap[12074]: modprobe: FATAL: Module zram not found in directory /lib/modules/5.13.1-xanmod-llvm-native
    Jul 10 13:26:33 debian root[12075]: Error: inserting the zram kernel module
    Jul 10 13:26:33 debian zramswap[12075]: <13>Jul 10 13:26:33 root: Error: inserting the zram kernel module
    Jul 10 13:26:33 debian systemd[1]: zramswap.service: Main process exited, code=exited, status=1/FAILURE
    So did your zram fail because dkms used gcc to try and compile it instead of "cc" aliasing clang?

    my vbox module is getting explicitly compiled by dkms with gcc and failing. switching cc to clang didn't do anything.

    I think this is a dkms issue. DKMS needs to use "cc" instead of explicit compiler calls to /usr/bin/gcc and it suite of tools

    No one is going to use llvm clang lto'ed kernels if this isn't sorted (other than those using openMandriva).

    Leave a comment:


  • reba
    replied
    Originally posted by DanglingPointer View Post
    Thanks in advance!
    Hi there. I just re-played the stuff in my posting and made some adjustments.
    For example bison was missing as a dependency and I added some snippets and explainations.
    And, yes, you're supposed to edit the Makefile of the linux headers.

    The module zramswap also breaks *sigh* so there might be others as well that are in need for a recompile.
    Maybe, after all, I am doing something wrong?
    Code:
    Jul 10 13:26:33 debian root[12072]: Starting Zram
    Jul 10 13:26:33 debian zramswap[12072]: <13>Jul 10 13:26:33 root: Starting Zram
    Jul 10 13:26:33 debian zramswap[12074]: modprobe: FATAL: Module zram not found in directory /lib/modules/5.13.1-xanmod-llvm-native
    Jul 10 13:26:33 debian root[12075]: Error: inserting the zram kernel module
    Jul 10 13:26:33 debian zramswap[12075]: <13>Jul 10 13:26:33 root: Error: inserting the zram kernel module
    Jul 10 13:26:33 debian systemd[1]: zramswap.service: Main process exited, code=exited, status=1/FAILURE

    Leave a comment:


  • DanglingPointer
    replied
    Originally posted by reba View Post
    Virtualbox will not work as the vboxdrv-module isn't loaded and has to be compiled manually.
    I made aptitude reinstall the virtualbox-dkms package but it fails when compiling for the active kernel.

    So I thought "Hm. Maybe the CC (which is usually gcc) for compiling the module does not like the Clang compiled kernel. So, what about compiling the module with Clang, too?"

    So I edited the Makefile of the vboxdrv to add the same compiler-options as for the kernel:
    Code:
    nano /usr/src/linux-headers-5.10.7-xanmod-llvm-native
    Just added these on top:
    Code:
    CC=clang
    HOSTCC=clang
    LLVM=1
    and the recompilation of the module worked!

    Trying
    Code:
    modprobe vboxdrv
    loaded the module without complains.
    Hi reba
    I have a similar problem where my vbox v6.1.22 fails to run any guests and its because the vboxdrv fails to compile through dkms.
    I naturally thought it was something for Virtualbox to fix so I filed a bug there... https://www.virtualbox.org/ticket/20425

    Not sure really who needs to fix this but it needs to get fixed, how are kernel modules going to get compile dynamically when changed if the Linux kernel is built with clang and lto=thin?

    How did you fix it? You mentioned you edited the Makefile of the vboxdrv but your quote above does a "nano" of the linux-header's Makefile? I'm a little confused and would appreciate if you could elaborate as I too would like to use clang lto'ed kernel. I've rolled back to gcc built kernel as I can't work without virtualbox.

    Thanks in advance!

    Leave a comment:


  • reba
    replied
    Originally posted by leandrolnh View Post

    That's the discussion of the news: LTO with Clang is not in mainstream yet. I don't think xanmod has added support for it either.
    That's why you haven't found it.
    Yes. I thought there were in but not yet enabled. Looks like they're just about to get in. It somehow reads they're just about to be enabled for 5.12.
    I don't think xanmod would do this amount of work downstream but rely on upstream.

    Anyways, my bod^M script is ready.

    Leave a comment:


  • aspen
    replied
    Originally posted by xpris View Post

    OpenMandriva ships Clang kernel as option in past release and Clang kernel should be default one in upcoming. So not that hard.
    Huh, you got a link to code that shows how they do it? I'm interested.

    Leave a comment:


  • leandrolnh
    replied
    Originally posted by reba
    Edit:
    At least for this 5.10.7
    Code:
    grep -R -i "LTO_CLANG" linux
    did not return any hits.
    That's the discussion of the news: LTO with Clang is not in mainstream yet. I don't think xanmod has added support for it either.
    That's why you haven't found it.

    Leave a comment:


  • reba
    replied
    Originally posted by AdelKS View Post
    reba I think you missed one more option to give to `make`, which is `LLVM_IAS=1`. If you don't use it, something is not enabled. I don't know exactly what.
    Yes, I've seen it but I don't know what this "integrated assembler" is supposed to mean?

    Originally posted by AdelKS View Post
    Edit: the patch says to do the following, to enable Thin LTO for example. I think you didn't enable LTO at all.

    $ make LLVM=1 LLVM_IAS=1 defconfig
    $ scripts/config -e LTO_CLANG_THIN
    $ make LLVM=1 LLVM_IAS=1

    And when LTO is enabled, the compilation will most probably fail with objtool errors, you can't compile a "normal" config file for now.

    Edit 2: You haven't said at all that that's how you enable LTO haha, I misread, my bad.
    No, LTO and GPO are not enabled AFAIK in my setup. That would be step two

    Their documentation mentions some parameters regarding LTO:
    Code:
    -flto, -fno-lto
    Enable LTO in ‘full’ mode
    -flto-jobs=<arg>
    Controls the backend parallelism of -flto=thin (default of 0 means the number of threads will be derived from the number of CPUs detected)
    -flto=<arg>
    Set LTO mode to either ‘full’ or ‘thin’
    As I said, that's step two - I was just awed Clang and LLVM work AT ALL
    Kernel compilation sadly takes some real time and there's only that much in a day.


    Edit:
    At least for this 5.10.7
    Code:
    grep -R -i "LTO_CLANG" linux
    did not return any hits.
    Last edited by reba; 13 January 2021, 05:17 PM.

    Leave a comment:


  • AdelKS
    replied
    reba I think you missed one more option to give to `make`, which is `LLVM_IAS=1`. If you don't use it, something is not enabled. I don't know exactly what.

    Edit: the patch says to do the following, to enable Thin LTO for example. I think you didn't enable LTO at all.

    $ make LLVM=1 LLVM_IAS=1 defconfig
    $ scripts/config -e LTO_CLANG_THIN
    $ make LLVM=1 LLVM_IAS=1

    And when LTO is enabled, the compilation will most probably fail with objtool errors, you can't compile a "normal" config file for now.

    Edit 2: You haven't said at all that that's how you enable LTO haha, I misread, my bad.
    Last edited by AdelKS; 13 January 2021, 04:35 PM.

    Leave a comment:


  • reba
    replied
    After reading the article I was tempted to try out my own Clang+LLVM compiled kernel (from xanmod) and... what can I say, it compiled beautifully and I'm currently running it

    Even the VirtualBox module could be compiled and the VMs are running without a problem! O_O

    If you're interested here's the script I came up with:
    Code:
    #!/bin/bash
    
    echo 'Installing the build environment...'
    echo 'See: https://wiki.debian.org/BuildADebianKernelPackage "Install the Required Packages"'
    # sudo apt-get install build-essential linux-source bc kmod cpio flex libncurses5-dev libelf-dev libssl-dev
    sudo apt-get install build-essential bison bc kmod cpio flex libncurses5-dev libelf-dev libssl-dev
    
    echo 'Installing Clang...'
    echo 'See: https://www.kernel.org/doc/html/v5.7/kbuild/llvm.html'
    sudo apt-get install clang
    
    echo 'Installing LLVM and LLD...'
    sudo apt-get install llvm lld
    
    # echo 'Extracting the kernel source to the current directory and changing into...'
    # echo 'See: https://wiki.debian.org/BuildADebianKernelPackage "Extracting the Kernel Source"'
    # tar xavf /usr/src/linux-source-*.tar.xz
    # cd linux-source-*
    
    echo 'Cloning xanmod kernel...'
    git clone https://github.com/xanmod/linux.git
    cd linux
    
    echo 'Setting environment for "make" to use Clang, LLVM, march, etc...'
    echo 'See: https://unix.stackexchange.com/a/545385'
    export CC="clang"
    export HOSTCC="clang"
    export LLVM="1"
    export KCFLAGS="-O2 -march=native -pipe"
    export KCPPFLAGS="-O2 -march=native -pipe"
    
    # First run only, otherwise this would overwrite custom settings
    # echo 'Configuring the kernel...'
    # echo 'See: https://wiki.debian.org/BuildADebianKernelPackage "Configuring the Kernel"'
    # make defconfig
    
    echo 'Setting new "localversion"...'
    echo "-xanmod-llvm-native" > localversion
    
    echo 'Building the kernel to a deb-package...'
    echo 'See: https://wiki.debian.org/BuildADebianKernelPackage "Building the Debian Package"'
    time nice make -j `nproc` bindeb-pkg
    Then just install the packages:
    Code:
    sudo dpkg -i ../linux-*.deb
    After rebooting the kernel can be selected from GRUB.

    Virtualbox will not work as the vboxdrv-module isn't loaded and has to be compiled manually.
    I reinstalled the whole virtualbox-dkms package but it fails when compiling for the active kernel:
    Code:
    sudo apt-get --reinstall install virtualbox-dkms
    
    ...
    Unpacking virtualbox-dkms (6.1.22-dfsg-2) over (6.1.22-dfsg-2) ...
    Setting up virtualbox-dkms (6.1.22-dfsg-2) ...
    Loading new virtualbox-6.1.22 DKMS files...
    Building for 5.13.1-xanmod-llvm-native
    Building initial module for 5.13.1-xanmod-llvm-native
    Error! Bad return status for module build on kernel: 5.13.1-xanmod-llvm-native (x86_64)
    Consult /var/lib/dkms/virtualbox/6.1.22/build/make.log for more information.
    dpkg: error processing package virtualbox-dkms (--configure):
    installed virtualbox-dkms package post-installation script subprocess returned error exit status 10
    Errors were encountered while processing:
    virtualbox-dkms
    E: Sub-process /usr/bin/dpkg returned an error code (1)
    So I thought: "Hm. Maybe the CC (which is usually gcc) for compiling the module does not like the Clang compiled kernel. So, what about compiling the module with Clang, too?"

    So I edited the Makefile of the vboxdrv to add the same compiler-options as for the kernel:
    Code:
    nano /usr/src/linux-headers-5.13.1-xanmod-llvm-native/Makefile
    Just added these line to the top of the file:
    Code:
    CC=clang
    HOSTCC=clang
    LLVM=1
    and the recompilation of the module worked:
    Code:
    sudo dpkg --configure virtualbox-dkms
    
    ...
    Loading new virtualbox-6.1.22 DKMS files...
    Building for 5.13.1-xanmod-llvm-native
    Building initial module for 5.13.1-xanmod-llvm-native
    Done.
    
    vboxdrv.ko:
    Running module version sanity check.
    - Original module
    - No original module exists within this kernel
    - Installation
    - Installing to /lib/modules/5.13.1-xanmod-llvm-native/updates/dkms/
    
    vboxnetadp.ko:
    Running module version sanity check.
    - Original module
    - No original module exists within this kernel
    - Installation
    - Installing to /lib/modules/5.13.1-xanmod-llvm-native/updates/dkms/
    
    vboxnetflt.ko:
    Running module version sanity check.
    - Original module
    - No original module exists within this kernel
    - Installation
    - Installing to /lib/modules/5.13.1-xanmod-llvm-native/updates/dkms/
    
    depmod...
    
    DKMS: install completed.
    Trying
    Code:
    modprobe vboxdrv
    loaded the module without complains:
    Code:
    $ lsmod | grep vbox
    vboxnetadp 28672 0
    vboxnetflt 32768 0
    vboxdrv 438272 2 vboxnetadp,vboxnetflt

    I am totally astounded how easy this was, even more so coming for a noob that I am, wowzers!

    Big kudos and thumbs up from me to everyone who contributed to make this whole thing absolutely seamless!
    Code:
    $ uname -a
    Linux debian 5.13.1-xanmod-llvm-native #1 SMP Sat Jul 10 12:23:59 CEST 2021 x86_64 GNU/Linux
    Last edited by reba; 10 July 2021, 07:15 AM. Reason: add more snippets, explictitly name the Makefile

    Leave a comment:

Working...
X