Build2 v0.13 Released As C/C++ Build Toolchain Inspired By Rust's Cargo

[andreano]

This conversion of foreign build systems (just so you can use them as dependencies) is again same strategy as Meson:
https://wrapdb.mesonbuild.com/

There is bound to be some duplication of effort here.

[boris-kolpackov]

You mean if every project uses the same build-system? And if we find some kinks in build2 that could be done better we covert everything to build2++.

I don't blame you for being skeptical. The history of software build systems is basically one hack built on top of another, starting with the original make (read on the choice of TAB if you haven't already), to automake, to CMake, etc. In contrast, build2 is an attempt to learn from all the mistakes and do it properly, no crapshots. And if we get it right, maybe we will see the same thing happening to build systems as what happened to version control systems. One thing for sure, 5 years down the road when we (hopefully) have C++20 modules, I don't want to be still writing CMakeFile's.

Those typically don't convert the build-system as that's rather high maintenance to keep in sync.

We've done our share of conversions on some non-trivial projects (OpenSSL, ICU) and this hasn't been our experience. Yes, initial conversion can be difficult, but after that, upgrading from one version to another is pretty easy. I don't know about you, but I would rather spend a couple of days and get it right than being constantly distracted by issues stemming from brittleness of the existing build system.

[discordian]

But can you actually use them without jumping through hoops? For example, is it possible to build the OpenSSL package from source on Windows out of the box? While there are short-term advantages of the Conan approach (which is to use whatever build system each packages comes with), there is also the long-term disadvantage of building a shiny castle on a shaky foundation. You can't blame some of us for wanting (and working towards) a different future.

You mean if every project uses the same build-system? And if we find some kinks in build2 that could be done better we covert everything to build2++. Sorry if I got you wrong, but that's what you seem to imply (there's some irony wanting to support only one build-system and then that being yet another one).
For Windows/Mac I'd guess that you rather prefer binaries anyway, Linux does not have ABI compatibility throughout distros hence those funny issues.

This only works to an extent, right? If you find yourself spending more and more time dealing with random packaging issues and less on actually writing your application (which I see quite a few reports of, when it comes to Conan), then the whole thing falls apart.

Hows build2's package manager immune to that, what if some package I need is not there, or a new version breaks your build2 integration?

The reality is that there are projects with tons of packages and alot of effort make everything work (conan, buildroot, yocto). Those typically don't convert the build-system as that's rather high maintenance to keep in sync.

[andreano]

you could ask why meson reinvented cmake
ok, now it's clear that you don't know: meson is meta build system for generating build system files, build2 is complete build system

I run ninja in build directories 100 times a day, so I think I know that. It's just that giving up my beloved 2-stage build process interests me very very little. I find great pleasure in seeing ninja recompile a large codebase in 0.000s and having multiple build directories in obscure places, so leaving that would just take extra convincing for me.

[kpedersen]

Or just don't fetch this package with conan if you do a local build. If its not a local build you will have to trick your way around anyway, sou you just could do that once and upload the binaries to your local repo - then you can just transparently use your binary.

Currently this is quite messy with CMake so I honestly would like build2 to provide something better here. So this part "tricking your way around" is exactly the part which if build2 can provide something elegant here, I think they will be onto a winner. Perhaps a decent adapter system or something like that or even something much simpler like a lookup table of binary libraries based on platform. The emcmake wrapper approach works (just!) but is not brilliantly nice.

[discordian]

160k? Is that even enough for just Debian x86 and Debian amd64? What about Debian aarch or even an older release of Debian x86? 160k seems a little low if they are ever going to hope to avoid falling back to source compilation.

Cant build the world + humans in less than 6 days or so.

The bit in bold is the part where I believe there is a weakness. As mentioned, most (if not all) platforms require patches to make source build. Unlike languages that just wrap C and don't need to worry about any real cross platform complexities, this is far more difficult with C based languages that need to do real work.

I.e pretend we have a dependence on Qt5. Conan or build2 will result in the following error:

Linker error: Undefined reference to procstat_open_sysctl
Linker error: Undefined reference to kvm_open

Because it is at the very least missing the patches from here:

https://svnweb.freebsd.org/ports/hea...t5-core/files/

In particular this part of the patch:

--- src/corelib/io/io.pri.orig 2018-02-08 18:24:48 UTC
+++ src/corelib/io/io.pri
@@ -178,6 +178,9 @@ win32 {
SOURCES += \
io/qstandardpaths_unix.cpp \
io/qstorageinfo_unix.cpp
+ freebsd {
+ LIBS += -lkvm -lprocstat
+ }
}

Note that this is generally not an issue for NPM, Crates.io, CPAN and others because they have no cross platform capabilities. They simply call into C libraries that does the hard stuff.

if the package doesn't support the target you need patches, sure. Get them upstream, get them into the conan package, or use a local repository with the patches.
Or just don't fetch this package with conan if you do a local build. If its not a local build you will have to trick your way around anyway, sou you just could do that once and upload the binaries to your local repo - then you can just transparently use your binary.

[boris-kolpackov]

It's a matter of whats available, vs starting fom scratch, if you dont use conan you need to do something else (everything manually?).
All the packages you mentioned are available, shows of a healthy community effort.

But can you actually use them without jumping through hoops? For example, is it possible to build the OpenSSL package from source on Windows out of the box? While there are short-term advantages of the Conan approach (which is to use whatever build system each packages comes with), there is also the long-term disadvantage of building a shiny castle on a shaky foundation. You can't blame some of us for wanting (and working towards) a different future.

Find issues? report or fix them [...]

This only works to an extent, right? If you find yourself spending more and more time dealing with random packaging issues and less on actually writing your application (which I see quite a few reports of, when it comes to Conan), then the whole thing falls apart.

[kpedersen]

i.e. you would replace package manager with your hands? what's next? you don't need toolchain because you could type binaries in hex editor?

If the package manager doesn't work and all I have left are my hands... then yes, I would. What would you do? Sit there and wait for someone else to do it for you?

Likewise if the toolchain is defective and all I had was a hex editor, hell yeah. I have deadlines to reach

But certainly the point is to discuss potential issues (and maybe come up with a solution) *before* the toolchain reaches a defective state.

[rene]

Could you elaborate on this? Specifically, what annoying parts of Cargo do you believe were carried over to build2? I am genuinely curious because we tried hard not to repeat some of the major (IMO, of course) mistakes of Cargo, such as lumping the build system and the package/project manager into a single tool or only supporting build by convention without a general build system (which leads to "build scrips" in the form of build.rs). Thanks in advance!

Annoying in Cargo is it's working against Linux distribution packaging, with Rust & Cargo it is currently quite impossible to package Linux distribution packages that install system-wide components and packages. Each time you build something a Rust / Cargo package re-builds it's hundreds of micro dependency. Due to that even maintaining a sane Firefox or librsvg package became a nightmare: https://www.youtube.com/watch?v=3MdTu8dSorY

[kpedersen]

Well, https://conan.io/center/ has 160k binary packages, and with conan there is also a source-package. Means either it can fetch the compiled library, or it can fetch its source and dependencies and build the lib locally. Then it also allows you to use local caches and local servers. So if your product has a battle-tested version with special-sauce compiler flags, it can use that prepared binary always, on multiple computers and multiple projects.

160k? Is that even enough for just Debian x86 and Debian amd64? What about Debian aarch or even an older release of Debian x86? 160k seems a little low if they are ever going to hope to avoid falling back to source compilation.

The bit in bold is the part where I believe there is a weakness. As mentioned, most (if not all) platforms require patches to make source build. Unlike languages that just wrap C and don't need to worry about any real cross platform complexities, this is far more difficult with C based languages that need to do real work.

I.e pretend we have a dependence on Qt5. Conan or build2 will result in the following error:

Linker error: Undefined reference to procstat_open_sysctl
Linker error: Undefined reference to kvm_open

Because it is at the very least missing the patches from here:

https://svnweb.freebsd.org/ports/hea...t5-core/files/

In particular this part of the patch:

--- src/corelib/io/io.pri.orig 2018-02-08 18:24:48 UTC
+++ src/corelib/io/io.pri
@@ -178,6 +178,9 @@ win32 {
SOURCES += \
io/qstandardpaths_unix.cpp \
io/qstorageinfo_unix.cpp
+ freebsd {
+ LIBS += -lkvm -lprocstat
+ }
}

Note that this is generally not an issue for NPM, Crates.io, CPAN and others because they have no cross platform capabilities. They simply call into C libraries that does the hard stuff.