OpenMandriva Is Finding Great Success In Their Switch To Using LLVM's Clang Compiler

[berolinux]

Except for licensing, I don't understand why would a package builder permanently switch from GCC to Clang (or vice versa) for all packages. Neither of the two compilers dominates the other one.

We decided to switch to clang mostly because we find its code to be more readable. gcc isn't "bad code" or anything, but let's say it has a very steep learning curve, while a regular developer who isn't a 100% compiler guy can usually work out how some parts of clang work by looking at its code.

It is true that you could probably build an even better performing system by mixing the compilers, picking whatever compiler compiles a particular package (or even file inside the package) best -- but that would be a bit of a maintenance nightmare (is the choice for every package still correct after an update of one of the compilers? Is the compiler that was picked for x86_64 also best for ARM? Is the same compiler still the best after deciding to drop support for truly ancient x86 CPUs by enabling e.g. SSE?).

[berolinux]

That is actually a good question. Since I’m not involved in the project I can only guess as to possible reasons. Some possibilities:

Very good guesses. Pretty much all of them had some part in the decision.

1. Better debugging and tools lead to catching more bugs. Simply using a different compiler can turn up questionable code.

Indeed -- I initially started trying to build every package with clang (before we switched to it as the main compiler, obviously) was that clang pointed out a number of errors in a particularly bad 3rd party bluetooth driver that gcc accepted (and built into something that did what the code said, but not quite what the person who wrote it meant).

Stuff like

void connectTo(char name[16]) {
   if(memcmp(name, something, sizeof(name)))
      ...

(where the incorrect assumption was that sizeof(name) would be 16 -- in fact, it's the platform's pointer size because a char[16] is passed as a char*...)

Making sure stuff compiles with multiple compilers helps catch errors, and clang had a number of sanitizers first. gcc did a great job catching up however.

Another point (the biggest, actually) that goes with "better tools" is "more readable code". It's not a secret that we don't have any of the gcc core developers on our team - it's also not a secret that gcc's code base has a very steep learning curve, it's hard to figure out how all of it works. With clang it's much easier for a developer who mostly works on other code to figure out how it all works and how it goes together. That part is still true today.

2. There was a mention of compiler speed but I’m not sure the delta even exists anymore. There may still be advantages in the overall development cycle.

Compiler speed was a lot better when we decided to switch (and given we aren't backed by big money, we don't have build machines with 128 cores and 256 GB RAM or so). This, too, is less of an issue today, gcc got faster and clang got slower (not by getting worse, but by adding more optimizations resulting in better code - that's a tradeoff we're certainly willing to make).

3. Faster code? This is debatable but I wouldn’t be surprised that modern libs help a lot here.

Depends on the particular package. Some packages perform better when built with clang, some perform better when built with gcc.
Unfortunately we can't make use of some llvm bits that would make things faster (e.g. libc++) because of binary compatibility concerns.

4. I’ve ever seen an issue being raised by the distro developers here that indicate an issue with licenses or even politics for that matter. They seem to be driven by a desire to build a better distro. Now given that I can see users caring not to have GCC and the encumbered libs as this reduces risk for developers.

Our license policy is, essentially, "If it's FOSS, it's good -- if it isn't, we don't want it" -- no problems with the GPL, no problems with BSD-ish licenses either.

By default we release stuff we write ourselves under the latest version of the GPL, but that may vary (e.g. if we were to write a clang plugin, we'd probably use the clang license to make it possible for clang to bundle it).

5. Maybe they like the challenge or just being different! Either way these alternative distro just strengthen Linux by increased by the diversity of solutions.

That, too, was part of it -- "let's see if it can be done" is always a good challenge.

At some point I'll probably start doing another build of all packages on top of the musl libc and libc++ -- that probably won't make the main distro because of binary compatibility concerns, but could be interesting for some embedded devices/special use cases where that doesn't matter.

[cybertraveler]

What does that make a system like, say, OpenMandriva that does use a number of GNU components, but goes a different way on a number of other components (clang compiler, KDE desktop, ...)? At what point does it stop being GNU/Linux and turning into something else?
Is it the use of gcc? glibc? coreutils?
I like the GNU project, but the GNU/Linux naming for something that isn't "the GNU OS with the Linux kernel", even if it relies heavily on some GNU components, not so much.

I don't see why you are creating confusion.

GNU started a project to make a "fully free" OS. They were successful after adding the Linux kernel to it. Since that time many people built upon, swapped out parts and added to GNU/Linux. I'm not gonna call a system GNU/KDE/X/Eclipse/Steam/Linux. It's a GNU/Linux distro with stuff changed. I don't even care that GNU started it other than the fact I appreciate they did so. I don't care what bits were added or changed. The system needs a name & the system has a name. That name is GNU/Linux.

It's not Linux. Linux is a kernel that is also used in non-GNU/Linux operating systems.

If you're here on this forum you should know all this. So why are you generating confusion? There is no need.

Noobs call it "Linux". They do not know the truth. They only know a word bounded around and popularised by other fools. They live in confusion. They may not even be aware that Android contains the Linux kernel, yet is not GNU/Linux and makes no attempt to be GNU/Linux. If they are aware of some vague relationship between Android and distros like Ubuntu, Fedora, Arch and Debian, they may not understand what that relationship is. Those same noobs may be unaware that GNU is also used with other kernels. If they are aware of that, they may struggling to understand this concept as in their minds they would be trying to formulate a concept of "Linux without Linux".

If some day Fedora is built only with CLANG & ships only with KDE, systemd and almost no GNU software, then perhaps you might like to simply call it Fedora and not called it a GNU/Linux distribution. Until that time comes, there is no need for further discussion or debate. The world is complicated enough without adding unnecessary complexity.

Please do not be foolish and do not spread confusion.

[skeevy420]

I don't see why you are creating confusion.

GNU started a project to make a "fully free" OS. They were successful after adding the Linux kernel to it. Since that time many people built upon, swapped out parts and added to GNU/Linux. I'm not gonna call a system GNU/KDE/X/Eclipse/Steam/Linux. It's a GNU/Linux distro with stuff changed. I don't even care that GNU started it other than the fact I appreciate they did so. I don't care what bits were added or changed. The system needs a name & the system has a name. That name is GNU/Linux.

If some day Fedora is built only with CLANG & ships only with KDE, systemd and almost no GNU software, then perhaps you might like to simply call it Fedora and not called it a GNU/Linux distribution. Until that time comes, there is no need for further discussion or debate. The world is complicated enough without adding unnecessary complexity.

Please do not be foolish and do not spread confusion.

What's funny is that Gnome is technically the GNU desktop and uses systemd.

I personally don't call it GNU/Linux because at some point adding the userspace tools to the name of the OS is just retarded.

I really don't want to tell people that I run: acpi/acpid/adriconf-git/alsa-firmware/alsa-utils/amdgpu-pro-vulkan-only/amdvlk/android-tools/android-udev/appimagelauncher/ark/aurutils/autoconf/automake/avahi/b43-fwcutter/bash/beanshell/binutils/bison/bluedevil/breath-dark-icon-theme/breeze-gtk/btrfs-progs/bzip2/calf/calligra/clazy/clementine-qt5-git/cmake/coin-or-mp/coreutils/cppcheck/cpupower/crda/cronie/cryptsetup/cups/cups-pdf/cups-pk-helper/device-mapper/devtools/dhclient/dhcpcd/diffutils/dmidecode/dmraid/dnsmasq/dolphin/dolphin-plugins/dosbox/dosfstools/e2fsprogs/ecm-tools/ecryptfs-utils/efibootmgr/exfat-utils/extra-cmake-modules/f2fs-tools/fakeroot/faudio-git/ffmpeg/ffmpegthumbs/file/filelight/filesystem/findutils/firefox/firefox-extension-video-download-helper/flatpak/flex/gamemode-git/gawk/gcc/gcc-libs/gdb/gettext/ghostscript/gimp-gtk3/git/glibc/gnome-icon-theme/gnome-themes-extra/grep/grub/grub-customizer/grub-theme-manjaro/gsfonts/gst-libav/gst-plugins-bad/gst-plugins-base/gst-plugins-good/gst-plugins-ugly/gtk-theme-breath/gtk3-classic/gvfs/gvfs-afc/gvfs-gphoto2/gvfs-mtp/gvfs-nfs/gvfs-smb/gwenview/gzip/hardinfo/haveged/heaptrack/hplip/htop/i-nex/illyria-wallpaper/imagewriter/inetutils/intel-ucode/inxi/iproute2/iptables/iputils/ipw2100-fw/ipw2200-fw/jfsutils/kaccounts-providers/kamera/kate/kcalc/kde-applications-meta/kde-gtk-config/kde-servicemenus-rootactions/kdeaccessibility-meta/kdeadmin-meta/kdebase-meta/kdeconnect/kdeedu-data/kdeedu-meta/kdegames-meta/kdegraphics-meta/kdegraphics-thumbnailers/kdemultimedia-meta/kdenetwork-filesharing/kdenetwork-meta/kdepim-meta/kdeplasma-addons/kdesdk-meta/kdeutils-meta/kdevelop/kdevelop-clang-tidy/kdevelop-pg-qt/kdevelop-php/kdevelop-python/kdewebdev-meta/kdiff3/keditbookmarks/kfind/kgamma5/kget/khelpcenter/kimageformats/kinfocenter/kio-extras/kmenuedit/konsole/konversation/kscreen/kscreenlocker/ksshaskpass/ksysguard/ksystemlog/kwallet-pam/kwalletmanager/kwayland-integration/kwin/kwrited/less/lib32-amdvlk/lib32-faudio-git/lib32-flex/lib32-gamemode-git/lib32-libcanberra/lib32-libcanberra-pulse/lib32-libva-intel-driver/lib32-libva-mesa-driver/lib32-libva-vdpau-driver/lib32-llvm/lib32-mesa-demos/lib32-mesa-vdpau/lib32-simplescreenrecorder/lib32-spirv-tools-tkg-git/lib32-vkd3d-tkg-git/lib32-vulkan-intel/lib32-vulkan-radeon/libcanberra/libcanberra-gstreamer/libcanberra-pulse/libdvdcss/libheif/libidn11/libktorrent/libmythes/libratbag/libreoffice-fresh/libstrangle-git/libtool/libva-intel-driver/libva-mesa-driver/libva-vdpau-driver/libwpg/licenses/linux-firmware/linux50/linux50-broadcom-wl/linux50-headers/linux50-rtl8723bu/linux50-tkg-bmq/linux50-tkg-bmq-headers/lldb/logrotate/lrzip/lsb-release/lsp-plugins/lutris/lvm2/lzop/m4/make/man-db/man-pages/manjaro-alsa/manjaro-browser-settings/manjaro-documentation-en/manjaro-firmware/manjaro-hello/manjaro-hotfixes/manjaro-kde-settings/manjaro-pulse/manjaro-release/manjaro-settings-manager-kcm/manjaro-settings-manager-knotifier/manjaro-system/manjaro-wallpapers-18.0/mda.lv2/mdadm/meld/memtest86+/mesa-demos/mesa-vdpau/mhwd/mhwd-db/milou/mkinitcpio-openswap/mobile-broadband-provider-info/modemmanager/mplayer/ms-office-online/mtpfs/nano/networkmanager/networkmanager-openconnect/networkmanager-openvpn/networkmanager-pptp/networkmanager-vpnc/nfs-utils/noto-fonts/noto-fonts-emoji/nss-mdns/ntfs-3g/ntp/numlockx/octopi/octopi-cachecleaner/octopi-notifier-frameworks/octopi-repoeditor/okteta/okular/opencl-amd/openresolv/openssh/opus-tools/os-prober/p7zip/pacaur/pacman/partitionmanager/patch/patchutils/pciutils/perl/perl-file-mimeinfo/phonon-qt5-gstreamer/piper/pkgconf/plasma-browser-integration/plasma-desktop/plasma-nm/plasma-pa/plasma-workspace/plasma-workspace-wallpapers/plasma5-themes-breath/poppler-data/postgresql-libs/powerdevil/powertop/print-manager/procps-ng/projectm-git/psmisc/pstoedit/pulseaudio-bluetooth/pulseaudio-ctl/pulseaudio-zeroconf/pulseeffects/pyqt5-common/python-pillow/python-pip/python-pyqt5/python-pysmbc/python-reportlab/qt5-doc/qt5-imageformats/qt5-virtualkeyboard/reiserfsprogs/repo/rsync/rubberband/ruby/s-nail/samba/sc-controller-git/screenfetch/sddm/sddm-breath-theme/sddm-kcm/sed/shadow/simplescreenrecorder/skanlite/smplayer/smplayer-skins/smplayer-themes/smtube/spectacle/spectre-meltdown-checker/spirv-headers-git/spirv-llvm-translator/spirv-tools-tkg-git/splix/sshfs/steam-manjaro/steam-native/steam-native-pcre-fix/subversion/sudo/sysfsutils/system-config-printer/systemd-fsck-silent/systemd-kcm/systemd-swap/systemd-sysvcompat/systemsettings/tar/telepathy-kde-meta/terminus-font/texinfo/thunderbird/tlp/ttf-bitstream-vera/ttf-droid/ttf-inconsolata/ttf-indic-otf/ttf-liberation/udiskie/udisks2/unarchiver/unixodbc/usb_modeswitch/usbutils/user-manager/util-linux/vdhcoapp/vi/vkd3d-tkg-git/vlc-nightly/vulkan-intel/vulkan-radeon/vulkan-tools/wallpapers-2018/wattman-gtk-git/wget/which/wine-tkg-staging-esync-d9+dxvk-faudio-marchopt-lutris-git/wine-tkg-staging-esync-vkd3d-d9+dxvk-faudio-marchopt-git/wine-tkg-staging-esync-vkd3d-d9+dxvk-faudio-marchopt-lutris-git/winetricks-tkg-git/wpa_supplicant/xdg-su/xdg-user-dirs/xdg-utils/xf86-input-elographics/xf86-input-evdev/xf86-input-keyboard/xf86-input-libinput/xf86-input-mouse/xf86-input-void/xf86-video-amdgpu/xf86-video-ati/xf86-video-intel/xf86-video-nouveau/xfsprogs/xorg-server/xorg-twm/xorg-xinit/xorg-xkill/yakuake-git/yelp/youtube-dl/zam-plugins/zfs-dkms-rc/zfs-utils-rc/zsh/GNU/Linux

[computerquip]

We decided to switch to clang mostly because we find its code to be more readable. gcc isn't "bad code" or anything, but let's say it has a very steep learning curve, while a regular developer who isn't a 100% compiler guy can usually work out how some parts of clang work by looking at its code.

It is true that you could probably build an even better performing system by mixing the compilers, picking whatever compiler compiles a particular package (or even file inside the package) best -- but that would be a bit of a maintenance nightmare (is the choice for every package still correct after an update of one of the compilers? Is the compiler that was picked for x86_64 also best for ARM? Is the same compiler still the best after deciding to drop support for truly ancient x86 CPUs by enabling e.g. SSE?).

If I'm honest, what reason would a package manager need to look at compiler code for? I find that to be a very odd reason to do anything from an end-user perspective.

EDIT: If you mean the assembly output, I don't get that either since I don't experience that at all, especially if you turn off optimization. I hope you don't mean the optimized assembly in clang is easier to read.

[berolinux]

If I'm honest, what reason would a package manager need to look at compiler code for?

In an ideal world, we'd have hundreds of thousands of contributors, so if someone building package A and runs into a compiler bug while doing that, he can just tell a compiler guy what happened, and will get an instant fix so he can continue his work on A.

In the real world, especially in a small all-volunteer project, someone building package A and running into a compiler bug will usually try to generate a small reproducer, report the bug (to the upstream project, which may not be super-interested in package A and may have different priorities), and if there's no response for a while look at the compiler code himself to figure out what's going wrong, and hopefully attach a patch to the bug report.

A really good package maintainer isn't scared of fixing up a library and/or tool [including compiler] used by his package if necessary (of course with the help of other contributors if needed).

[berolinux]

GNU started a project to make a "fully free" OS. They were successful after adding the Linux kernel to it.

Agreed up to that point.

Since that time many people built upon, swapped out parts and added to GNU/Linux. I'm not gonna call a system GNU/KDE/X/Eclipse/Steam/Linux. It's a GNU/Linux distro with stuff changed. I don't even care that GNU started it other than the fact I appreciate they did so. I don't care what bits were added or changed. The system needs a name & the system has a name. That name is GNU/Linux.

That is where the actual confusion starts. At what point does a "system with swapped out parts" stop being "GNU/Linux" then? Does it keep the name when it swapped out everything?

They may not even be aware that Android contains the Linux kernel, yet is not GNU/Linux and makes no attempt to be GNU/Linux.

So going by that idea, Android (which is built using GNU make, GNU awk, GNU bash and several other GNU tools) is not GNU/Linux, but another Linux based system that uses some GNU tools but diverges greatly from GNU in other parts is?
Where do you draw the line? Making an official attempt to be GNU/Linux? (That would make a lot of distributions that you call GNU/Linux something other than GNU/Linux)? Keeping the idea of a "fully free OS"? (That would make the AOSP subset of Android GNU/Linux, but a gaming distribution that is essentially the GNU OS with steam bundled on top of it not)?
Binary compatibility? (That would essentially make it "if it uses glibc, it's GNU")?

If some day Fedora is built only with CLANG & ships only with KDE, systemd and almost no GNU software, then perhaps you might like to simply call it Fedora and not called it a GNU/Linux distribution. Until that time comes, there is no need for further discussion or debate.

That time has essentially come. If I look at a stock OpenMandriva installation, it has glibc, bash, grep, awk, sed, coreutils and a few more GNU tools. It also shares the same idea of a fully free OS. But it goes its own way on lots of just about everything else. It uses clang as main compiler. Increasingly llvm tools replace parts of GNU binutils. KDE as the only desktop in a default installation. LXQt as a secondary supported desktop. tar from libarchive (also not because GNU tar is bad, but because libarchive tar has some features we like a lot -- such as being able to operate on zip files as well). rpm for package management. We usually prefer cmake over autoconf.
We love the idea of the GNU OS ("100% free OS"), but aren't too fond of much of its implementation (don't get me wrong, there's some great parts in it, but there's also a lot of technical decisions that can be disagreed with and some code that's not all that good), so we're building what we hope is a better implementation of the same idea, using the same components where GNU's ideas and ours are the same even on the technical/implementation side.
We don't pay attention to ideas written down in the GNU coding standards we disagree with (such as using guile for extensibility, texinfo for documentation, etc., just to name a few).

The world is complicated enough without adding unnecessary complexity.

Please do not be foolish and do not spread confusion.

Agreed about that too -- just not about the idea that calling a Linux based OS that also includes some GNU components something other than GNU/Linux is spreading confusion.
It's far more confusing to insist on using GNU/Linux without defining at what point something stops being GNU/Linux and starts being something else.

[berolinux]

Hi. Have you also tried to run (or been thinking about running) both compilers on the same file and throwing away one of the results, to potentially get more warning messages about code quality?

Purely from a warning/error message perspective, that makes sense -- but it also doubles compile times even when doing a simple rebuild without any code changes, so we aren't doing that. We do build important stuff both compilers once in a while to see if anything suspicious comes up.

[Ironmask]

I don't see why you are creating confusion.

GNU started a project to make a "fully free" OS. They were successful after adding the Linux kernel to it. Since that time many people built upon, swapped out parts and added to GNU/Linux. I'm not gonna call a system GNU/KDE/X/Eclipse/Steam/Linux. It's a GNU/Linux distro with stuff changed. I don't even care that GNU started it other than the fact I appreciate they did so. I don't care what bits were added or changed. The system needs a name & the system has a name. That name is GNU/Linux.

It's not Linux. Linux is a kernel that is also used in non-GNU/Linux operating systems.

If you're here on this forum you should know all this. So why are you generating confusion? There is no need.

Noobs call it "Linux". They do not know the truth. They only know a word bounded around and popularised by other fools. They live in confusion. They may not even be aware that Android contains the Linux kernel, yet is not GNU/Linux and makes no attempt to be GNU/Linux. If they are aware of some vague relationship between Android and distros like Ubuntu, Fedora, Arch and Debian, they may not understand what that relationship is. Those same noobs may be unaware that GNU is also used with other kernels. If they are aware of that, they may struggling to understand this concept as in their minds they would be trying to formulate a concept of "Linux without Linux".

If some day Fedora is built only with CLANG & ships only with KDE, systemd and almost no GNU software, then perhaps you might like to simply call it Fedora and not called it a GNU/Linux distribution. Until that time comes, there is no need for further discussion or debate. The world is complicated enough without adding unnecessary complexity.

Please do not be foolish and do not spread confusion.

Where is the confusion? Everyone knows Linux is a kernel. They're referring to a common set of functionality that's often added on top of it. Time and time again, people can say "why not just call it every package on your system and be done with it". I've never seen that point contested, only ignored. Even in your very own post, you preemptively brought that argument up, then ignored it by saying "well it's just confusing so you shouldn't do it, you bad bad man." If someone can consistently bring up a counter point to your argument then your argument, regardless of intention and how correct that intent is, is in itself incorrect, or rather just structured very poorly. It's a basic point of idiomatic English, you're telling people that natural speech is wrong and they should be willing themselves into an unclear and intuitive naming scheme. That sounds a lot more like "unnecessary complexity" to me. What's idiomatic to people is calling a system Linux when it contains the Linux kernel and a standard C library. You can't argue against what people already think is idiomatic, you're outright saying the world is incorrect, I don't care if it actually is right or not. You need to sympathize with people to construct an argument they'll listen to. Calling me a "noob" and a "fool" will make me ignore you and your argument outright, because it's a sign to me that you can't empathize with me and thusly you're not even aware of how to properly construct an argument that relates to my situation.

People disregard your argument of "It's not Linux it's GNU with Linux added" because it feels like you're trying to take away Linux from them. They care about Linux more than GNU, they can replace the GNU parts but they can't or don't want to replace Linux. You saying "No, it's GNU, end of story" has a similar tone to them as "You live under my roof, you live by my rules". Linux users are often defiant and rebels, they hate being told what to do or what is or isn't right. Your lack of communication skills is not helping that. Every time you try to try to push the horse from behind, it's desire to buck only increases. You offer no argument to the naming scheme other than "you have to do it by law or else". No.

If a specific distribution calls itself Linux, then I say it's Linux. If they call themselves GNU/Linux, then I use that. I use Ubuntu Linux while you might use Purism GNU/Linux. I respect the naming scheme of the project as I would like to be respected for my decisions. I don't try to enforce my own naming scheme on people or projects who have done more work than I did. If a project named itself Systemd/GNU/Linux I'd call it that too, no matter how stupid that naming scheme is. And going along with your argument, a lot of distros should be called that, since Systemd is such a critical part of the system now, a lot of packages won't even work without it anymore. After all, don't want to "confuse" people using Void thinking they can run Systemd-depending packages. But I don't have to concern myself with that silly naming scheme because I respect the naming schemes of the projects that utilize Linux and whatever packages they choose to use on top of it, whether they use Systemd, or glibc, or not. Not saying you have to, in fact I'm not saying you have to do anything, except you have to stop telling me what I have to do, which I consider pretty reasonable. Feel free to call it FreeBSD/kFreeBSD, after all, don't want to confuse people who might be using kFreeBSD with glibc installed. I'm not sure why GNU stuff gets to magically append it's name to any project while others can't, according to your logic, but, it's your life, do what you want, I'm not going to call you a "fool" for doing so, because I know you have your own reasoning for doing things. You and your opinions are not any more invalid than mine, until the second you insist mine are invalid.

[computerquip]

In an ideal world, we'd have hundreds of thousands of contributors, so if someone building package A and runs into a compiler bug while doing that, he can just tell a compiler guy what happened, and will get an instant fix so he can continue his work on A.

In the real world, especially in a small all-volunteer project, someone building package A and running into a compiler bug will usually try to generate a small reproducer, report the bug (to the upstream project, which may not be super-interested in package A and may have different priorities), and if there's no response for a while look at the compiler code himself to figure out what's going wrong, and hopefully attach a patch to the bug report.

A really good package maintainer isn't scared of fixing up a library and/or tool [including compiler] used by his package if necessary (of course with the help of other contributors if needed).

I mean, I get that but that's a very rare case. Certainly not reason enough to change an entire package base in order to not use gcc, right?