Libjpeg-Turbo 2.0 Beta Brings More AVX2 SIMD, Improved CMake Build System

[cybertraveler]

Autotool-generated configure, on the other hand, will certainly NOT work on Windows. It's a shell script, it requires a real shell. CMD.exe is definitely not one.
You'll need to install either Cygwin (userland POSIX) or WSL aka Bash on Windows (an alternative Linux-like subset API for the NT kernel) first.

Something worth considering there is that the POSIX shell standard and BaSH are both very lightweight languages and interpreters compared to Python. One of Python's slogans is "batteries included"; IE a full Python environment has libraries in it for doing many, many different common programming tasks.

Your own comment also somewhat nullifies your point. Windows is already a bloated target platform and this bloated platform has readily available support for shell scripts via Cygwin and WSL (as you said). So it's a non-issue.

Finally: Meson's design does not require Python. They simply implemented Meson in Python. In the future they or someone else may re-implement it in something else. Theoretically, you could implement it in pure shell; not that I'm suggesting that. Remember that the Meson build scripts aren't Python, they are Python-like. You could write the build script interpreter in any programming language and the act of doing so would not be like writing a full Python implementation or interpreter.

[DrYak]

Something worth considering there is that the POSIX shell standard and BaSH are both very lightweight languages and interpreters compared to Python.

I wouldn't call a shell that not only supports regular expressions but even features a working network stack "lightweight".

But yes indeed it's bootstrapable very early in a system without tons of dependencies.

(NOTE: I'm only pointing things out. I'm not really criticising Bash. Yes, it's a bit overblown, but I like the features and it comes handy in a few situations.).

Windows is already a bloated target platform and this bloated platform has readily available support for shell scripts via Cygwin and WSL (as you said). So it's a non-issue.

I'm mostly pointing out details about the history of CMake.

Kitware started it as a way to configure project such as VTK/ITK on not only typicall Unices/POSIX systems, but even on standard Windows which has nearly nothing in common with Unix (and I think maybe pre-X versions of Mac OS)

The point was a "one ring to rule them all" tool that could build your project not only using your typical Unix stack (make, cc, etc.) but also using Visual C CLI tools.

So, to someone coming from the Unix world, CMake seems sometime weird, relying on weird trick and some hard coded path to try to look for libraries, mostly because Windows lacks a standard tools like 'pkg-config'.


That's why you can have a shell-only-dependent autotool project, but cmake/meson requiring porject.

A standard POSIX systems comes more or less with its own set of batteries that a simple shell script can leverage (but at the cost of needing to pay attention to varying dialects, and differing small features. Hence the contrived shell code often found in configure, and the huge proportion of time lost at looking at stuff such as mixed-word orders, missing string C functions, etc.)

Windows has none, and CMake and Meson need to bring their own batteries.

Finally: Meson's design does not require Python. They simply implemented Meson in Python.

Well, yeah, to be more correct I should have said :
"Meson projects require a functionning Meson present, whose only current implementation is written in Python and thus requires the presence of Python"

In the future they or someone else may re-implement it in something else. Theoretically, you could implement it in pure shell; not that I'm suggesting that.

If somebody tries that, they'll probably end up re-inventing M4 :-P

...and some troll is bound to suggest re-implementing it in Brainfuck.

[Vistaus]

Like I said, with one key difference. With meson, the above requires meson to be presently installed on the computer (and thus python, too).

Yes, I've read and liked your entire post before posting my reply (which wasn't even aimed at you, my aim was to complement carewolf's post).

[DRC42]

I can speak to this topic, as the one who ultimately had to make the decision vis-a-vis libjpeg-turbo (after a lot of discussion: https://github.com/libjpeg-turbo/lib...rbo/issues/226). Autotools makes sense when you're developing a system library that will mainly be used/developed on Un*x and that contains only C/C++ code. When you step outside of that paradigm, it starts to make less sense. Autotools requires Cygwin or MSYS on Windows, even if all you're doing is just running the configure script, and both of those implementations have really slow file I/O in my experience. Cygwin is worse than MSYS, but even the MSYS implementation runs configure scripts so slowly as to be practically unusable for me-- particularly over a network-attached drive. That is perhaps not a huge problem for those who just want to build the library once and install it, but anyone who wants to actually work on the code has to build the library many times a day. I believe in making it as easy as possible for as many people as possible to experiment with the code, and thus libjpeg-turbo has always used a different build system for Windows. I rolled my own at first, then I switched the Windows builds to CMake in libjpeg-turbo 1.1. CMake makes the most sense for Windows, IMHO, because it allows downstream developers to generate Visual C++ projects (which most Windows developers are more accustomed to than Makefiles), but it also frees the maintainer (me) from having to maintain Visual C++ project files for a variety of different Visual C++ releases. Furthermore, it allows developers to use MinGW Makefiles or NMake Makefiles if they prefer those compilers/build systems. CMake also greatly reduces the build requirements for the Windows platform (simply install the CMake application rather than an entire Un*x workalike system.) On Mac, autotools is plenty fast, but Mac developers are also not [as] accustomed to that type of development, and using autotools on that platform (if you're doing anything other than just configuring and building the project from a pre-configured tarball) requires MacPorts or Homebrew.

The primary justification for switching to CMake had less to do with whether autotools was inherently better or worse and had more to do with choosing a build system that made the most sense, in the aggregate, for all of the platforms and languages we have to support. It was challenging to develop proper autotools build rules for our Java and assembly code, as well as to implement our in-tree packaging and unit testing system with autotools. It required a lot of hacks that CMake/CTest largely eliminated. Also, there are some cases in which just running ./configure will not suffice, even if you're building from a tarball rather than Git. For instance, building libjpeg-turbo <= 1.5.x for iOS required that the configure script be generated with a relatively recent autotools release, but I also had to build our "official" Linux binaries on an older Linux distro (usually the last supported CentOS release) in order to make them cross-distribution compatible. At the time, CentOS 5 was our official build platform (pre-Docker), but the release of autotools on that platform was too old to generate a configure script that worked for iOS builds. Thus I had to go to the trouble of building a newer autotools release from source and pointing my official builds to that newer release. But-- no surprise-- the newest autoconf/automake wouldn't work with the older M4 and libtool, so I had to build those as well.

I would say that, when autotools "just works", there is a good argument to be made for it being a more intuitive build system for Un*x platforms, but when it doesn't "just work", it's a royal PITA. If someone wants to just build and install the library for their specific platform, using a pristine source tarball, then invoking './configure; make install' is *usually* (but not always) sufficient, but anyone who wants to get their hands dirty with the code will probably have to install autotools, and depending on what they're trying to do, they may or may not be able to use the version of autotools that is pre-installed on their Un*x system. Outside of the context of libjpeg-turbo, it never failed that I would check out a library from a Git repository (usually because I needed a bug fix that wasn't available in the latest tarball), but the autotools-based build system for that library required a newer autotools release than the one installed on my build platform. The myth is that you can download the source, run './configure; make install', and everything will work fine. The reality is that that only works if your users are building from a release tarball, and sometimes it doesn't even work in that case. For library maintainers, autotools effectively puts us in the uncomfortable position of either eschewing new autotools features, in favor of backward compatibility with older build platforms, or eschewing older build platforms. Neither is very attractive, because for every build platform that doesn't work with our build system, I have to field multiple inquiries from the community as to why. Projects that use autotools are inherently less cross-platform/cross-distribution compatible because autotools is inherently less cross-platform/cross-distribution compatible. With CMake, I can just specify the minimum version required in order to support the CMake features we use, and I always know that it's easy for developers to download and install that minimum version, even if their O/S doesn't provide it (well, easy as long as they're using Windows, Mac, or Linux-- less easy for other Un*x platforms, on which CMake has to be built from source. Everything's a trade-off.)

CMake's support for GNU-style constructs, such as setting libdir, bindir, etc., is admittedly awkward, but newer versions include GNUInstallDirs.cmake, which makes it less awkward. libjpeg-turbo includes a version of that script with my own modifications to make it even more GNU-like (I haven't checked whether those modifications were ever accepted upstream.) That highlights another advantage of CMake, IMHO-- it's somewhat easier to extend than autotools is (at least for me.) With autotools, you have to become familiar with several different languages: shell scripts, M4, and GNU Make. CMake just has one, and whereas CMake syntax is less intuitive for me than writing a shell script, it's a lot more intuitive than M4. Passing --libdir, --prefix, etc., to the configure script is an expectation on Un*x platforms (particularly GNU platforms) but not others (Windows and Mac don't have the same concept of central library and executable directories-- at least not ones that are meant for non-O/S use.) So yes, autotools, as a GNU build system, does generally dovetail more nicely with GNU platforms, but not so much with other platforms.

Many system libraries support both CMake and autotools, but given the complexity of our build/packaging/testing system and the need to support Java and assembly language sources, this just didn't make sense from a project maintainer's point of view. It was difficult enough maintaining two build systems when one (autotools) was limited to Un*x only and the other (CMake) was limited to Windows only. Maintaining two cross-platform build systems was untenable. Don't get me wrong-- CMake has many annoyances, and for years, I fought against migrating our Un*x build system to it. In the final analysis, though, it just made the most sense for our project. I also kept receiving requests from downstream projects that used CMake (thanks to VTK and Qt and whatnot, CMake is popular among graphics applications) and wanted to include libjpeg-turbo in-tree. Thus, there were effectively several bastardized libjpeg-turbo CMake build systems out there already, and people kept begging me to integrate one of those. However, I wasn't willing to do that unless the CMake build system could cover all of the features of the autotools build system. I was ultimately going to have to support a full cross-platform CMake build system in order to make the community happy, and supporting that in addition to the autotools build system would have created a lot of additional unpaid work for me.

All of that aside, I don't claim that what made sense for libjpeg-turbo makes sense for everybody. Every project is different. There is good and bad about both autotools and CMake.

DRC