On Built Module Interface Compatibility

or, Testing a bunch of build systems
which claim to support modules

In this post we'll cover a common problem among build systems which claim to "support modules". If you want to jump into the technical stuff and not me waxing philosophic, skip to the first heading.

Translating from one language to another, unless it is from Greek and Latin, the queens of all languages, is like looking at Flemish tapestries from the wrong side, for although the figures are visible, they are covered by threads that obscure them, and cannot be seen with the smoothness and color of the right side.

— Cervantes, Don Quixote

Domain expertise is a funny thing. Before developing some expertise in a domain, we often judge tools from that domain based on their ease-of-use. Think of the controls a consumer road car compared to an Formula One race car. The consumer car is “better” because it’s intuitive to use, teenagers get the hang of it in a few weeks. If all you need is to do commute from one place to another, these controls and the machine they’re attached to are perfectly servicable.

However, once domain expertise has been established, we often stop thinking in terms of ease-of-use and begin thinking in terms of capabilities. The Formula One car is “better” because it enables behavior the road car can’t achieve. A non-expert driver can’t get the car around a single corner, but that’s not what the domain expert is concerned with.

I work on build systems all day. Developing them, writing build scripts for projects, porting projects from one to the other, so on and so forth. For better or worse, I have domain expertise in build systems. I’m chiefly concerned with what they can do, and how much code is required to make them do it, not with how simple or intuitive their interfaces are. I want to talk about something some build systems can do, and other can’t. Not the interface, the capability.

The hot new capability in C++ build systems is modules. It is an easy way for new entrants and mavericks to distinguish themselves from the old guard. “… and we support C++20 modules!” is in almost every ReadMe.md for these kinds of build systems.

However, there’s a reason modules took so long to be adopted by the build systems of yesteryear; they’re difficult to implement, and shortcuts break builds. Many build systems are taking shortcuts with modules. In this post we explore a common way shortcuts break: Built Module Interface Compatibility.

An Introduction to Built Module Interfaces

In pre-modules C++ we had headers and implementation files. These combined to form translation units (TUs), which the compiler chunked through to produce object files. Object files are hardy mechanisms; their formats are standardized by platform vendors, and anything not controlled by the vendors is dictated by ABI standards. They are broadly interchangeable, object files produced by one compiler can be combined with object files from another by a linker.1

Modules complicate this story. Modules ship as collections of module units, each of which is a TU in and of itself.2 Every module unit needs to be built, every module unit produces an object file. Some module units represent interfaces. Like the headers of yore, these contain declarations which other TUs want to use.

We cannot access these declarations via plain text inclusion, like with headers, nor do the object files contain the information we need.3 Instead the compiler produces a sidecar when building these module units which describes the unit’s interface. This is the Built Module Interface, or BMI.

A build system will pass these BMIs to the compiler whenever it is building a TU which needs the declarations they contain. Achieving this alone is a bit of work for the build system and language machinery, because naively neither is aware of which declarations are produced by a given TU, or which TUs want to consume those declarations. Succeeding at this is the bar most build systems set for “supporting modules”.4

The subtlety comes from the BMIs themselves. Where object files are hardy, BMIs are extremely sensitive. Compatibility between compilers is out of the question. Compatibility between different builds of the same compiler is dubious at best. And here’s the kicker: compatibility between different invocations of the same build of the same compiler is not guaranteed, it’s not even particularly likely.

A guaranteed way to break BMI compatibility is to change language standards. If the producer of a BMI is compiled with C++23, and the consumer wants to use C++26, that’s a build failure. To correctly build the project the build system needs to recognize these incompatibilities and reconstruct BMIs under flags compatible with consumers. In my opinion, this is the bar for “supporting modules”.

The Test

In order to judge build system support for managaging BMI compatibility, we need some common project under which we will define said support. Our example will be four build system targets, a provider and three consumers. The provider builds under C++23, one consumer also builds under C++23, and the remaining two build under C++26. This will test three things:

We don’t need anything fancy here on the language side, two files will do it. One will export a C++ module, the other will import it. The importing file can be reused for all three consumers. For completeness, the code is:

// provider.cppm
export module provider;

// consumer.cpp
import provider;

The tested build systems will be the major cross-platform players which claim any level of modules support: Bazel, CMake, and Xmake. As well as some mavericks and brand-new build systems which claim the same: build2,5 Qbs, pcons, and cppbuild. I’ll be using the latest-at-time-of-writing trunk for each.

The complete code is available here.

The Quixotic Attempts

Unsurprisingly, all the mavericks and newcomers failed the test. Surprisingly, so did Bazel, and it fails hard. Bazel’s module support is experimental, so this isn’t a slight against it. We’ll dispatch with analyzing the others first and then come back to Bazel.

build2 is unsurprising here. It’s known not to support BMI compatibility and has a comment where the BMI compatibility check would go noting this. I would like to see this pothole more clearly sign-posted in the documentation, rather than being arcana for build engineers who read implementation code.

Qbs’s module support is documented as experimental, so no harm in failing. The only differentiator being I can’t find any infrastructure in the code where compatibility would even be implemented, no stubs, no TODOs.

pcons is so new it still has the new-build-system-smell. It is the only candidate which is openly vibe-coded. If you’re wondering if AI can save you from BMI-compatibility, the answer is No. Like Qbs, there doesn’t appear to be any notion of compatibility at all.6 pcons is also the only system considered which performs collation only once at configure time,7 but it’s brand new so we’ll forgive it.

Calling cppbuild new would be unfair, new implies complete, it’s in-construction. However the author has been dragging modules work forward by pushing for the libraries cppbuild relies on to provide module units, so it gets a shout out. Unfortunately it doesn’t handle BMI compatibility yet, but I’m sure it will soon.

So, Bazel. Again, module support is experimental which means there’s no commitment to this stuff working. It fails the BMI compatibility test, but the bigger problem is it doesn’t even support GCC. This is due to a broken template in rules_cc which hasn’t been meaningfully changed since it was written in 2024 and I don’t believe ever worked. It sends the preprocessor output to the dependency file path it wants, and writes the dependency file output to a temp file which is discarded.

This is a trivial-to-fix bug, but it means no one is testing Bazel’s module support on GCC. This thing breaks on Hello World. Comprehensive testing of modules support is absolutely essential for production build systems, so there remains work to do before moving out of experimental.

The Victors?

CMake and Xmake both produce successful builds for the test, but neither passes on all the listed criteria. CMake rebuilds the BMI for every single consumer, and Xmake rebuilds the BMI for each incompatible consumer. This is more work than needs to be done.

CMake’s behavior is due to a stubbed out calculation for BMI compatibility which uses the consumer’s name as a compatibility hash instead of anything having to do with standards versions or flags. This is fixed in a pending MR for CMake 4.4 and with that MR applied CMake passes on all criteria.8

Xmake doesn’t maintain a cache of available BMIs the way CMake does, each consumer asks a binary question: Is this consumer compatible with the provider? If not, the BMI is rebuilt by the consumer asking the question. Xmake also doesn’t reuse scans, each target scans and collates its entire graph. This is simple and easy to debug, but wasteful. Rescanning behavior is unique to Xmake, none of the other build systems considered rescan module units.

Another interesting behavior of Xmake is the discriminate_on_defines policy. This policy determines if definition flags are considered for the purposes of BMI compatibility calculation. This is something of a strange question, what does it matter if the provider and consumer have different compile definitions?

It matters because…

Nobody Knows How to Rebuild BMIs

I’ve lied to you reader. No build system handles BMI compatibility correctly, because there is no generally agreed correct way to rebuild BMIs. There are several very serious toolchain engineers who hold that C++ modules require all code involved in an entire application, including any dynamic libraries it uses, be built under precisely identical compile flags.9 That the very question of BMI compatibility is ill-formed. I do not agree with them, but the problem is hard.

Consider the following provider:

// provider.cppm
module;

#include <private_header.hpp>

export module provider;

We have a private header, a header which is meant to only be accessible to the provider target. If a consumer needs to rebuild the BMI for this module unit, how does it get access to this header? More generally, when we rebuild a BMI, which flags get swapped out in order to make the rebuilt BMI compatible with the consumer, and which flags remain the same?

Our two victors disagree on the answer:

So the above example builds fine on CMake, and fails on Xmake. However, Xmake isn’t wrong, they had a motivated reason for this behavior: shared library exports. The Xmake bug covering this issue explicitly uses changing the compile definitions between the provider and consumer as a way to control symbol visibility, and they manifest “changing” by not propagating private includes and definitions to the consumer’s rebuild of the BMI.

The CMake bug covering the same issue is unresolved, and currently unsolvable within CMake outside truly heinous hacks like compile defintion smuggling. Personally, I believe a dedicated mechanism for this problem is better than Xmake’s “solution” of failing to rebuild BMIs with private headers.11

Coda

I don’t know man. Modules are really hard. An irrelevant microfraction of the C++ community seems to understand that, and even among toolchain people the problems are discounted. C++ implementers can solve hard problems. Two-phase lookup is hard, vague-linkage is hard, lots of things are hard, but worth it.

I think modules are really cool, I think these problems are totally solvable and will eventually result in a really good mechanism for the language. However, I also think the era of pretending the language is the only part of C++ which requires guidance and standardization is over.

We will not solve BMI compatibility and other problems like it while every build system is freestyling on the semantics of what “supporting modules” means. The two major working module implementations on the build system side have irreconcilable differences regarding how modules work. There must be guidance from the top or things will only get worse as implementations become more mature, and more ossified in that maturity.

#ReviveTheEcosystemIS

  1. This remains true unless we’re dealing link-time optimization, where the compiler uses object files as a carrier for compiler-specific IR. Exceptions abound in toolchain engineering. ↩︎

  2. After preprocessing. It’s still possible to combine module units with headers. ↩︎

  3. This should be self-evident, consider template declarations which otherwise make no appearance in object files. ↩︎

  4. All future references to “scanning” and “collation” (also known as “aggregation”) are about this step. The build system and the compiler collude to figure out which TUs provide which modules, and which TUs want to import those providers. ↩︎

  5. Please debate in your forum of choice if build2 is a maverick or a major build system. ↩︎

  6. This was my first time using pcons and I learned it has the unfortunate behavior of assuming if the same source file appears in multiple targets, it can reuse the resulting object file. This is obviously wrong, as I may have compiled it under different flags.

    If the author happens to read this: Hey, don’t do that. It’s a bad habit. ↩︎

  7. Meaning we can break the build by editing existing files to a different valid shape then rerunning Ninja. This is pretty catastrophic by build system standards. It leads to “just nuke the build folder and it starts working for some reason” bugs. ↩︎

  8. Full disclosure, it’s my MR. ↩︎

  9. I had a snarky joke here, but I’ve decided to replace it with: toolchain work is thankless drudgery which everyone hates you for doing. That this pile of chewing gum and ducktape works at all is a minor miracle. Next time you run into a compiler engineer, linker guru, or stdlib maintainer, buy them a beer. ↩︎

  10. CMake-speak for the language version standard. ↩︎

  11. I should note here, this fully precludes “everything builds under the same flags always”. We must provide some mechanism for consumers to recieve different flags than providers. The only place “same flags everywhere all the time” works is the MSVC toolchain, which “magically” translates __declspec(dllexport) into __declspec(dllimport) when consuming BMIs, presumably because they too didn’t want to solve this problem.

    However, this causes other problems, prompting the invention of the wonderfully obscure /dxifcSuppressDllImportTransform flag. I like to think of the ISDIT flag as a secret handshake among build system people. ↩︎