Skip to content

Cross-compilation support - take two - #12356

Open
andreabedini wants to merge 27 commits into
haskell:masterfrom
andreabedini:andrea/feature/cross-compilation-2
Open

andreabedini wants to merge 27 commits into
haskell:masterfrom
andreabedini:andrea/feature/cross-compilation-2

Conversation

@andreabedini

Copy link
Copy Markdown
Collaborator

This is a rewrite of #11179. Same goal, re-expressed from scratch on top of
current master as a reviewable series: 27 commits, each one buildable, with
the behaviour-changing ones separated from the refactors that make them
possible. #11179 can be closed in favour of this.

The feature is a second build stage. --with-build-compiler names the
compiler that produces programs which run on the machine doing the build --
custom Setup.hs scripts, their dependencies, and build-tool-depends --
as distinct from the host compiler named by --with-compiler, which produces
the artifacts you actually want. Without it, cabal has exactly one compiler
and silently assumes the two roles coincide.

The model

Stage = Build | Host names the system a package is built for.
Staged a = Staged { onHost :: a, onBuild :: Maybe a } holds one value per
stage, and the Maybe is the single source of truth for whether this is a
cross build: there is no separate flag to keep in sync. getStage resolves
Build to the host value when there is no distinct build stage, so every
per-stage lookup degenerates to the old host-only behaviour in an ordinary
build.

Almost every commit is a no-op off cross-compilation, by construction rather
than by testing, because a non-cross build has one stage and every getStage
returns the host toolchain. I hope this helps making the PR reviewable.

configureToolchains decides whether there is a second stage, and it
compares the configured compilers rather than the flags. sameCompiler
looks at compiler id, ABI tag and target platform (the properties that go
into a unit id) and ignores the ProgramDb. Passing
--with-build-compiler equal to --with-compiler therefore collapses to a
single stage. Treating it as a cross build would solve, elaborate and build
every shared dependency twice under indistinguishable UnitIds.

What changes

Solver:

  • PackagePath gains a Stage, orthogonal to Namespace and Qualifier,
    so the independent-goal machinery and the qualifier logic are untouched.
  • Index is keyed by stage, converted against each stage's own compiler and
    installed packages.
  • qualifyDeps flips toolStage to prevStage at the exe and setup tool
    boundaries when cross, so host packages depend on build-stage tools.
  • Goal linking and the single-instance restriction are keyed by stage. They
    were not, and the effect was a real solve failure, not a theoretical one:
    a local package that is also a build tool appears at both stages, the two
    goals get linked, and the solve dies with "cannot merge" and "multiple
    instances" on their different base.
  • Validation (extensions, languages, pkg-config) and
    configuredPackageProblems use the compiler of the stage each goal was
    solved for.

Install plan:

  • The elaborated plan is keyed by WithStage UnitId, so the host and build
    copies of a package are distinct nodes even when they share a unit id.
    This depends on the polymorphic-key install plan (refactor(cabal-install): generalise GenericInstallPlan to arbitrary node keys #12092, merged).
  • Each package is finalised and configured with its own stage's toolchain,
    and the stage compiler is hashed, so under distinct compilers the two
    copies get distinct unit ids.
  • The project's package-db settings stay with the host compiler.
    package-dbs: and --package-db name databases of the host compiler, and
    a package database is only readable by the compiler that wrote it, so
    handing them to a second compiler's ghc-pkg is wrong twice over: it can
    fail to read them, and if it does read them it offers host units to
    build-stage goals. The condition is whether a stage has a distinct
    toolchain, not whether it is the build stage — setup scripts are built at
    prevStage, which for a host-stage package is Build, so an ordinary
    non-cross build must keep the settings there.
  • Per-package package-db stacks, registration, and the running
    installed-package index are per stage. A merged index answers a lookup by
    name with units of the wrong stage, which is how this first showed up:
    Cabal's hsc2hs looks the rts up by name and dies with "No (or multiple)
    ghc rts package is registered".
  • BuildStatusMap and BuildOutcomes are keyed by the plan key. In-place
    unit ids are not compiler-specific, so the old projection onto bare
    UnitId kept one status for both copies and could mark a package
    installed while its dependencies were still configured.
  • plan.json entries carry a "stage" field, since the same unit id can
    legitimately appear twice.

Store:

  • Store locations are keyed by the platform the compiler targets as well as
    the compiler: store/<platform>/<compiler-id>-<abi>/<unit-id>, the same
    segment order distBuildDirectory has always used. The compiler id and
    ABI tag say which compiler built a package but not what it produces, so
    two compilers targeting different architectures were sharing a directory
    and a package database. This changes the layout for everyone; existing
    entries are not migrated or read, and are rebuilt on first use.

Testing

Four end-to-end tests, and a --with-build-compiler flag for the runner.
Three of them skip when only one GHC is available, CI included:

  • BuildCompilerOption, the flag is accepted and a build succeeds.
  • BuildCompilerSetup, a custom Setup.hs reports the
    __GLASGOW_HASKELL__ it was compiled with, which is the build compiler's.
  • BuildCompilerToolFromSource, a build tool and its library dependency
    built from source, checking which compiler each artifact came from.
  • BuildCompilerSameCompiler, the degenerate case above, where both
    stages name one compiler. Needs no second GHC, so it always runs.

I ran the following tests with host 9.12.2 and build 9.10.3 where a second
compiler was needed:

  • full cabal-testsuite, non-cross: 679 run, 25 skipped, 0 unexpected fails
  • cross e2e: 4/4
  • cabal-install unit tests: 642/642
  • cabal.validate.project with -Werror: clean
  • hlint 3.10 and fourmolu 0.12.0.0 on the touched files: clean

Not in this PR

  • User-facing syntax for stage-scoped constraints. The internal
    ConstraintScope is stage-aware, but there is no way to write a
    build:-qualified constraint, deliberately: the syntax should be settled
    together with the other constraint-syntax gaps (Constraint syntax is not expressive enough. #3502).
  • A way to name package databases for the build stage. There is currently
    no option for it: a distinct build toolchain gets its own global database
    and its own store, and nothing else.
  • Target selection (availableTargets, setRootTargets, TargetsMap) is
    still keyed by bare UnitId, which is why a package used as a build tool
    becomes a root at both stages under all.
  • Per-stage build directories, and the project-local store the GHC build
    wants. Those are a separate series.

The two types the rest of the cross-compile work is built on.

'Stage' = Build | Host names the system a package is built for: Build is
where the compiler runs, Host is where the artifacts it produces run. In
an ordinary build the two coincide.

'Staged a = Staged { onHost :: a, onBuild :: Maybe a }' is one value per
stage. The host value is mandatory, the build value is there only when
cross-compiling, so the value itself records whether this is a cross
build ('isCross') and there is no separate flag to keep in sync with it.
'getStage' resolves Build to the host value when there is no distinct
build stage, 'always x = Staged x Nothing' is the non-cross case, and
'activeStages' lists the stages carrying a distinct value.

A record rather than 'Stage -> a' so that Eq/Show/Binary/Structured
derive.

No consumers yet.
Groundwork for solving host and build packages together: both the
solver's paths and its installed index have to know which stage they
belong to.

'PackagePath' gains a third field, 'PackagePath Stage Namespace
Qualifier'. Stage is orthogonal to Namespace and Qualifier, so the
existing namespace/independent-goal machinery and the qualifier logic
(qBase/qSetup/inheritedQ) are left intact. Every path is still
constructed at Host; the Host->Build transition comes later in the
series, and 'toolStage' is a placeholder until then.

'Index' becomes 'Map Stage (Map PN (Map I PInfo))'. Installed packages
are per-stage by nature and a goal always knows its stage, so keying at
the top lets a lookup pick the stage's sub-index first. 'I' is unchanged
('I Ver Loc'), which leaves 'showI' and all solver rendering alone.
'convPIs' populates the Host stage only.

Both changes are additive: the 217 modular solver unit tests pass
unchanged.
Copilot AI lite review requested due to automatic review settings September 14, 2026 09:02
@andreabedini andreabedini mentioned this pull request Sep 14, 2026
6 tasks
'DependencyResolver' now takes the platform, compiler info and
installed-package index per stage, as 'Staged'; 'modularResolver'
consumes the Host stage. The only caller, 'resolveDependencies', wraps
its single toolchain with 'always', so a non-cross build behaves
identically. Prepares 'convPIs' to build a per-stage index, which the next commit
does.
…amDb)

'configureCompiler' returns a (Compiler, Platform, ProgramDb) triple that
the planning phases then thread around as three loose values. They
describe one configured compiler installation and are never useful apart,
so give them a name: 'Toolchain'. 'Toolchains = Staged Toolchain' is the
per-stage form cross-compilation needs, a host toolchain and a build
toolchain.

The module re-exports Distribution.Solver.Types.Stage so that consumers
import one module.

Nothing constructs a 'Toolchain' yet.
@andreabedini
andreabedini force-pushed the andrea/feature/cross-compilation-2 branch from cc82a12 to eaf4a1c Compare September 14, 2026 09:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical and moderate stage-handling issues remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds stage-aware cross-compilation support to cabal-install, separating build-machine tools from host-target artifacts across solving, planning, storage, execution, and tests.

Changes:

  • Adds Build/Host toolchains and stage-qualified solver and install-plan nodes.
  • Separates stores, package databases, indexes, statuses, and plan.json metadata by stage and platform.
  • Adds build-compiler options, documentation, and end-to-end tests.
File summaries
File Review
doc/nix-local-build.rst Reviewed; no final comments.
doc/cmd-v2-help/update.txt Reviewed; no final comments.
doc/cmd-v2-help/test.txt Reviewed; no final comments.
doc/cmd-v2-help/target.txt Reviewed; no final comments.
doc/cmd-v2-help/run.txt Reviewed; no final comments.
doc/cmd-v2-help/repl.txt Reviewed; no final comments.
doc/cmd-v2-help/path.txt Reviewed; no final comments.
doc/cmd-v2-help/outdated.txt Reviewed; no final comments.
doc/cmd-v2-help/list-bin.txt Reviewed; no final comments.
doc/cmd-v2-help/install.txt Reviewed; no final comments.
doc/cmd-v2-help/haddock.txt Reviewed; no final comments.
doc/cmd-v2-help/gen-bounds.txt Reviewed; no final comments.
doc/cmd-v2-help/freeze.txt Reviewed; no final comments.
doc/cmd-v2-help/exec.txt Reviewed; no final comments.
doc/cmd-v2-help/configure.txt Reviewed; no final comments.
doc/cmd-v2-help/build.txt Reviewed; no final comments.
doc/cmd-v2-help/bench.txt Reviewed; no final comments.
doc/cabal-project-description-file.rst Reviewed; no final comments.
doc/cabal-commands.rst Reviewed; no final comments.
changelog.d/with-build-compiler.md Reviewed; no final comments.
changelog.d/store-platform.md Reviewed; no final comments.
changelog.d/stage-keyed-install-plan.md Reviewed; no final comments.
changelog.d/solver-stage-foundation.md Reviewed; no final comments.
changelog.d/plan-json-stage.md Reviewed; no final comments.
changelog.d/build-with-own-toolchain.md Reviewed; no final comments.
cabal-testsuite/src/Test/Cabal/Prelude.hs nit (1 vote): Update the store-layout comment to reflect the new directory order.
cabal-testsuite/src/Test/Cabal/Plan.hs Reviewed; no final comments.
cabal-testsuite/src/Test/Cabal/Monad.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/WarnEarlyOverwrite/dirty-install.out Reviewed; no final comments.
cabal-testsuite/PackageTests/WarnEarlyOverwrite/clean-install-by-symlink.out Reviewed; no final comments.
cabal-testsuite/PackageTests/WarnEarlyOverwrite/clean-install-by-copy.out Reviewed; no final comments.
cabal-testsuite/PackageTests/Regression/T9756/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/Path/Compiler/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/Path/All/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/Install/ProgramAffixes/overwrite-policy.out Reviewed; no final comments.
cabal-testsuite/PackageTests/HaddockBuildDepends/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/ExtraPackages/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/ConditionalAndImport/cabal.out Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/tool/tool.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/tool/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/mylib/src/MyLib.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/mylib/mylib.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/cabal.test.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/cabal.project Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/app/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerToolFromSource/app/app.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSetup/client/Setup.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSetup/client/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSetup/client/client.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSetup/cabal.test.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSetup/cabal.project Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/tool/tool.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/tool/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/mylib/src/MyLib.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/mylib/mylib.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/cabal.test.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/cabal.project Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/app/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerSameCompiler/app/app.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerOption/hello/Main.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerOption/hello/hello.cabal Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerOption/cabal.test.hs Reviewed; no final comments.
cabal-testsuite/PackageTests/BuildCompilerOption/cabal.project Reviewed; no final comments.
cabal-testsuite/main/cabal-tests.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Solver/Modular/Solver.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Solver/Modular/QuickCheck.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Solver/Modular/DSL/TestCaseUtils.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Solver/Modular/DSL.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Client/Store.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Client/ProjectPlanning.hs Reviewed; no final comments.
cabal-install/tests/UnitTests/Distribution/Client/ProjectConfig.hs Reviewed; no final comments.
cabal-install/tests/IntegrationTests2.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Types/PackageSpecifier.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Toolchain.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Targets.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Store.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/SetupWrapper.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Setup.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ScriptUtils.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectPlanOutput.hs moderate (1 vote each): Preserve dependency stages in both executable-dependency forms; use each package’s stage platform for artifact suffixes in plan.json.
cabal-install/src/Distribution/Client/ProjectPlanning/Types.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectPlanning/Stage.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectOrchestration.hs moderate (1 vote): Build-stage reports use the host compiler and target platform; filter or construct reports per stage.
cabal-install/src/Distribution/Client/ProjectConfig/Types.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectConfig/Lens.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectConfig/Legacy.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectConfig/FieldGrammar.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectBuilding/UnpackedPackage.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectBuilding/Types.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/ProjectBuilding.hs moderate (1 vote): Derive job control from active stage compilers or use stage-specific controllers.
cabal-install/src/Distribution/Client/InstallPlan.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Install.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/InLibrary.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Get.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Freeze.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Fetch.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/DistDirLayout.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Dependency.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Configure.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/Config.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdRepl.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdPath.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdListBin.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdInstall.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdHaddockProject.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdHaddock.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdGenBounds.hs Reviewed; no final comments.
cabal-install/src/Distribution/Client/CmdExec.hs Reviewed; no final comments.
cabal-install/parser-tests/Tests/ParserTests.hs Reviewed; no final comments.
cabal-install/parser-tests/Tests/files/project-config-shared/cabal.project Reviewed; no final comments.
cabal-install/cabal-install.cabal Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/Stage.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/SolverPackage.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/SolverId.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/ResolverPackage.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/PackagePath.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/PackageConstraint.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/InstSolverPackage.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Types/DependencyResolver.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Validate.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Solver.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Preference.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Package.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Message.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Linking.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/IndexConversion.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Index.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Explore.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Dependency.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/ConfiguredConversion.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular/Builder.hs Reviewed; no final comments.
cabal-install-solver/src/Distribution/Solver/Modular.hs Reviewed; no final comments.
cabal-install-solver/cabal-install-solver.cabal Reviewed; no final comments.
Review details

Suppressed comments (6)

cabal-install/src/Distribution/Client/ProjectBuilding.hs:359

  • compiler here is the host compiler, but the resulting semaphore is passed to every package's Setup build. In a cross plan, build-stage packages invoke the build compiler; if it lacks JSEM support or uses an incompatible protocol, those packages still receive -jsem and can fail (while the opposite mismatch needlessly disables build-stage parallelism). Derive job control from all active stage compilers or use stage-specific controllers.
    cabal-install/src/Distribution/Client/ProjectOrchestration.hs:1297
  • fromPlanPackage emits reports for every configured plan node, including build-stage packages, but uses the single host comp and plat captured above. A build-stage artifact is therefore reported with the host compiler and target platform, and can be indistinguishable from the host copy. Filter build-stage nodes from host reports or construct and store reports per stage.
    cabal-install/src/Distribution/Client/ProjectPlanOutput.hs:206
  • This strips the stage from every executable dependency in plan.json, even though the same UnitId can occur in both stages. A consumer can therefore not resolve this edge to the correct plan entry (in particular, it cannot distinguish a host-stage internal executable from a build-stage build tool). Preserve the dependency stage in the JSON representation, with a schema that remains unambiguous when both entries share an id.
    cabal-install/src/Distribution/Client/ProjectPlanOutput.hs:219
  • The component-level form has the same ambiguity: withoutStage removes the stage from an executable dependency while the owning entry now explicitly supports Build and Host copies. When both stages contain the same UnitId, plan.json does not identify which node this edge targets. Emit the stage along with this dependency as well.
    cabal-install/src/Distribution/Client/ProjectPlanOutput.hs:170
  • The new stage field makes build-stage plan entries externally visible, but the artifact paths produced below still choose the extension from plat, which is always the host platform. For a cross target with a different executable or DLL suffix, plan.json will advertise a bin-file that does not exist for Build-stage entries. Use the package's stage platform (elabPlatform elaboratedSharedConfig elab) for both executable and foreign-library extensions.
    cabal-testsuite/src/Test/Cabal/Prelude.hs:1452
  • The new defaultStoreDirLayout writes store/<platform>/<compiler-id>-<abi>/, but this comment still documents the old directory order. The traversal below happens to handle the new layout; update the comment so the helper accurately describes the layout it is searching.
  • Files reviewed: 135/135 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

stageInplacePackageDbs :: Stage -> PackageDBStackCWD
stageInplacePackageDbs s =
stageCorePackageDbs s
++ [distPackageDB (compilerId (stageCompilerOf s))]
[ ( installedUnitId pkg
, elabOrderLibDependencies pkg
, elabOrderExeDependencies pkg
, map withoutStage (elabOrderExeDependencies pkg)
(compilerInfo comp)
pkgConfigDB
(fmap (\tc -> (compilerInfo (toolchainCompiler tc), toolchainPlatform tc)) toolchains)
(always pkgConfigDB)
@TeofilC

TeofilC commented Sep 14, 2026

Copy link
Copy Markdown
Member

I still believe it would be valuable to split this further and have separate PRs for each atomic change. That would be very valuable if they introduced a bug in the future.

I'm going to remove myself from the reviewers here, because I'm under the impression that you've used LLMs to some extent, and I do not review non-trivial patches created with the help of LLMs in my spare time.

@TeofilC
TeofilC removed their request for review September 14, 2026 09:59
…ectConfig

Let the user name a separate compiler for the build stage: the one that
produces programs which run on the machine doing the build (custom
Setup.hs scripts and their dependencies, build-tool-depends), as opposed
to the host compiler named by --with-compiler.

Command line (ConfigExFlags): '--with-build-compiler=PATH', short '-W',
alias '--with-build-hc'; and '--with-build-hc-pkg=PATH'. Both go through
~/.cabal/config (Config.hs combine) like their host counterparts.

Project file (ProjectConfigShared): 'build-compiler' (flavour, no command
line variant, mirroring 'compiler'), 'with-build-compiler' and
'with-build-hc-pkg', in both the FieldGrammar and the legacy grammar
(filterFields include list plus a hand-written build-compiler field).
Setting any of them inside a conditional is rejected, as for the host
compiler fields: the compiler has to be known before conditionals can be
evaluated. Lenses, Arbitrary and the parser tests are extended so the
fields round-trip.

Documented in cabal-project-description-file.rst and cabal-commands.rst;
the cmd-v2-help goldens are regenerated.

Deliberately not included: a way to name package databases for the build
stage. The project's 'package-dbs:' belong to the host compiler, and a
separate option for the build stage can follow.

Nothing consumes the new settings yet. 'configureToolchains' does.
Replace 'configureCompiler' with 'configureToolchains', which returns a
per-stage 'Toolchains'. The host compiler (--with-compiler) is configured
as before and under the same "compiler" cache file, so an existing
dist-newstyle is not reconfigured.

When a build compiler is requested (build-compiler,
--with-build-compiler or --with-build-hc-pkg) it is configured as a
second toolchain. Flavour and path fall back to the host's; hc-pkg does
not, since it belongs to one compiler installation.

That second toolchain is kept only if it really is a different compiler.
'sameCompiler' compares the properties that determine a unit id --
compiler id, ABI tag, target platform -- and ignores the ProgramDb, so
the same compiler named two ways ('ghc' and its absolute path) compares
equal. Treating "a build compiler was passed" as "this is a cross build"
would solve, elaborate and build every shared dependency twice under
indistinguishable UnitIds, and that is not hypothetical: GHC's own staged
bootstrap passes --with-build-compiler explicitly equal to
--with-compiler for its non-cross stage1.

With no build compiler requested, or one that coincides with the host's,
'onBuild' is Nothing: a non-cross build configures exactly one toolchain
and the build stage resolves to it through 'getStage'. That Nothing/Just
is the single source of truth for whether we are cross-compiling, which
is why it gets unit tests of its own
(UnitTests.Distribution.Client.ProjectPlanning).

'configureCompiler' stays as a thin Host-extracting wrapper, so the
elaboration phases, ScriptUtils and CmdPath need no changes.
Teach the runner about a build-stage compiler: '--with-build-compiler
PATH' is recorded in CommonArgs/TestEnv, and 'withBuildCompiler' runs a
test body with that path or skips the test when none was given. So these
tests are skipped in any configuration that has only one GHC, CI
included.

Two tests:

  - BuildCompilerOption: cabal accepts --with-build-compiler and still
    builds a trivial package.
  - BuildCompilerSetup: a custom Setup.hs injects the
    __GLASGOW_HASKELL__ it was compiled with into the executable it
    configures, and the test asserts that is the build compiler's
    version. That is the evidence the setup script was compiled by the
    build compiler rather than the host's.

At this point in the series nothing consumes the build toolchain yet, so
BuildCompilerSetup fails when run with --with-build-compiler. It starts
passing once installed packages are resolved per stage and setup scripts
use the build toolchain, later in the series.
'rebuildInstallPlan' collapsed to the host compiler at the
'configureCompiler' boundary. 'phaseConfigureCompiler' now returns the
full 'Toolchains' (via 'configureToolchains'), threaded through the
configure/programs/solver/elaborate phases and into 'planPackages'.
'elaborateInstallPlan' and 'planPackages' each take one 'Toolchains'
argument, dropping the Platform/Compiler/ProgramDb they already
contained.

The per-stage toolchains are flattened into 'ElaboratedSharedConfig': the
host scalars (pkgConfigCompiler/Platform/CompilerProgs) stay, and
build-stage counterparts
(pkgConfigBuildCompiler/BuildPlatform/BuildProgs) join them. No duplicate
storage, and no churn for host-toolchain consumers.

The solver ('resolveDependencies') and the actual use of the build
toolchain come later in the series; each phase extracts the host stage
for now.
'DependencyResolver' takes its toolchain inputs per stage:

  Staged (CompilerInfo, Platform) -> Staged (Maybe PkgConfigDb)
    -> Staged InstalledPackageIndex -> ...

Compiler info and platform are paired because they always come from the
same toolchain, and the pkg-config database joins the staged block since
a cross build has one per stage. Every caller wraps its single host
toolchain with 'always' and 'modularResolver' still consumes the Host
stage only, so behaviour is unchanged. The per-stage installed index
moves into the resolver params in the next commit.
'depResolverInstalledPkgIndex' becomes 'Staged InstalledPackageIndex'.
Keeping the index in the params next to the policy that edits it, rather
than threading it separately, is what lets the
'hideInstalledPackages*' helpers edit the host stage only (through the
new 'Stage.overStage'): packages hidden to force a rebuild are host-stage
packages, while the build stage keeps its installed packages, which are
reused from the build compiler.

'basicDepResolverParams' / 'basicInstallPolicy' / 'standardInstallPolicy'
take the index as 'Staged InstalledPackageIndex'; 'resolveDependencies'
and 'resolveWithoutDependencies' read it from the params, so there is no
separate index argument and no host-only transform applied at solve time.
All callers wrap their single index with 'always', and a non-cross build
is unaffected.

Adds 'Distribution.Solver.Types.Stage.overStage :: Stage -> (a -> a) ->
Staged a -> Staged a'.
Until now the stage was threaded as Host everywhere: structure without
effect. This makes the transition real.

  - 'convPIs' builds a genuine per-stage Index, one StageIndex per active
    stage ('activeStages' of the staged toolchains). The Build stage is
    converted against the build toolchain's compiler and installed
    packages, the Host stage against the host toolchain's. A non-cross
    build has only the host stage, so both the index and the behaviour
    are byte-identical.
  - 'defaultQualifyOptions' derives 'qoCross' from the index's stage
    keys -- a Build stage is present iff we are cross-compiling -- so the
    same single source of truth is read at the solver core, where the
    toolchains are not in scope.
  - 'qualifyDeps' flips 'toolStage' from 's' to 'prevStage s' at the
    exe/setup tool boundaries when cross, so host packages depend on
    build-stage tools. A non-cross build keeps tools at Host.

The per-stage index population and the Maybe-based 'Staged' it relies on
come from earlier commits in the series; this one is only the solver-side
transition.

Known gap, closed later in the series: the validation phase (Modular.hs,
for supported extensions/languages and the pkg-config database) and the
post-solve 'configuredPackageProblems' check still use the host toolchain
for Build-stage goals.
Completes what the FIXME left by the stage-transition commit flagged. Install-plan construction
resolves each pre-existing package against the installed-package index of
its own stage, cabal-install reads that index per stage from the staged
toolchain, and custom Setup.hs scripts are compiled and linked against
the build toolchain's compiler and package databases.

  - solver: 'convCP' takes a 'Staged InstalledPackageIndex' and looks a
    pre-existing package up in the index for its own stage, taken from
    the PackagePath.
  - cabal-install: 'phaseRunSolver' reads the installed index once per
    stage ('traverseWithStage' over the staged toolchain); pre-supplied
    host environment packages stay on the host stage. The build stage's
    index is read inside the cached solver-plan block, so every stage's
    toolchain is part of the cache key and changing only
    --with-build-compiler invalidates the cached plan.
  - cabal-install: the setup-script package-db stack and toolchain come
    from the build stage ('elabSetupPackageDBStack',
    'setupHsScriptOptions').

The project's package-db settings stay on the host stage.
'package-dbs:' and '--package-db' name databases of the host compiler,
and a package database is only readable by the compiler that wrote it, so
handing them to a second compiler's ghc-pkg is wrong twice over: it can
fail to read them, and if it does read them it offers host units to
build-stage goals. 'projectPackageDbsFor' gives a stage its share of
them, which for a distinct build toolchain is none -- its stack is its
own global database plus its own store.

The condition is whether the stage has a distinct toolchain, not whether
it is 'Build'. Setup scripts are built at the stage before the package's
own, and 'prevStage Host' is 'Build', so an ordinary non-cross build asks
for the Build stage on every build with a custom Setup and has to keep
the project's databases there. Unit tests pin both directions
(UnitTests.Distribution.Client.ProjectPlanning).

Adds 'Stage.traverseWithStage', which traverses a 'Staged' with the stage
each value belongs to.

All of it is a no-op for a non-cross build, where the build stage falls
back to the host toolchain. Verified with a distinct build compiler (host
9.12.2 / build 9.10.3): BuildCompilerSetup and BuildCompilerOption pass,
and the 217 Modular unit tests and the non-cross custom-setup tests are
unchanged.

Intermediate state, closed later in the series: packages are still
elaborated and registered with the host toolchain here, so a setup
dependency that had to be built from source under cross-compilation would
land in the host store. BuildCompilerSetup passes because its setup
dependencies are pre-existing in the build compiler's global package db.
Split 'ConstraintScope' into an optional build 'Stage' plus a
'ConstraintQualifier' (the former scope constructors). 'Nothing' matches
every stage; 'Just s' restricts the constraint to goals solved for stage
s. Top-level goals are always solved at Host, so 'scopeToplevel' pins the
stage to 'Just Host'; every other pre-existing constraint site keeps its
all-stage behaviour with 'Nothing'. Namespace matching for 'ScopeTarget'
(independent goals) is preserved.

This is the internal representation only. User constraints carry no stage
and are converted with 'Nothing', and the pretty-printer does not render
the stage: the only staged scopes that exist so far are the 'Just Host'
ones made by 'scopeToplevel', which render as before. Syntax for
stage-scoped user constraints is a separate change, so it can be settled
together with the other constraint-syntax gaps (haskell#3502).
The 'WithStage' wrapper and 'HasStage' class that will key the elaborated
install plan by build stage, so that the build-stage and host-stage
builds of one unit become distinct plan nodes.

'WithStage' is an 'IsNode' via 'Key (WithStage a) = WithStage (Key a)',
building on the polymorphic-key install plan (haskell#12092). Its 'Pretty'
instance renders the Host stage invisibly, so once the plan key becomes
'WithStage UnitId' the output of a non-cross build is unchanged.

Nothing references the module yet.
Replace the six flat host/build scalars (pkgConfig{,Build}{Platform,
Compiler,CompilerProgs/Progs}) with one grouped, staged
'pkgConfigToolchains :: Toolchains'. The compiler, platform and program
database of a stage are configured together and are meaningful together,
so they travel together; 'configureToolchains' already produces a
'Toolchains', which is now stored as one instead of destructured.

The former field names survive as host-/build-stage accessor functions
('pkgConfigCompiler = toolchainCompiler . getStage Host .
pkgConfigToolchains', and so on), so the ~60 plain readers are untouched
and only construction, record-pattern and record-wildcard sites change.
'setPkgConfigCompilerProgs' replaces the two ad-hoc host-progDb record
updates in haddock.

Behaviour-preserving: the plan is still elaborated against the host stage
everywhere. This is the regrouping that makes the per-stage selection,
'getStage (pkgConfigToolchains shared) elabStage', native rather than a
lookup through scalars.
Give elaboration enough stage information to configure each package with
the toolchain of the stage it belongs to. Plan-node identity does not
change yet; that comes with keying the plan by stage.

  - 'SolverPackage' and 'InstSolverPackage' gain a 'Stage', populated in
    'convCP' from the stage each node's PackagePath already carries.
  - 'ElaboratedConfiguredPackage' gains 'elabStage', set from the solver
    stage, and ProjectPlanning.Types gains 'elabToolchain' /
    'elabCompiler' / 'elabPlatform' / 'elabProgramDb': how to get a
    package's toolchain out of the shared config.
  - In 'elaborateSolverToCommon' each package is finalised (os/arch/impl
    conditionals) and configured against 'getStage toolchains elabStage'
    -- its own stage's compiler, platform and program database -- rather
    than the host-pinned toolchain. The call sites outside that function
    that used the host scalars for a specific package
    ('computeInstallDirs', 'packageHashConfigInputs',
    'setupHsConfigureFlags', 'setupHsHaddockFlags', 'elabDistDirParams',
    'normaliseConfiguredPackage') move to the elab* accessors. Hashing
    the stage compiler is what gives the host and build copies of a
    package distinct UnitIds, under cross-compilation with distinct
    compilers.
  - 'setupHsScriptOptions' picks its toolchain as
    'pkgConfigStageToolchain sharedConfig (prevStage elabStage)' instead
    of the flat build-stage accessors.
  - The host and build compilers stay bound at the top level for the
    project-wide computations (package DBs, supported build ways).

No-op for a non-cross build: with no separate build stage the build stage
falls back to the host toolchain (see 'getStage'), so every package
selects the toolchain it selected before.

Intermediate state: the per-package package-db stacks
('elabBuildPackageDBStack', 'elabRegisterPackageDBStack') still come from
the host-pinned corePackageDbs/inplacePackageDbs, and the library-way
selection is still host-pinned, which a later commit fixes. The plan is
still keyed by plain UnitId here; keying it by stage comes next.
Under cross-compilation the host and build copies of a package can share
a UnitId, so the plan graph has to tell them apart. Make it stage-aware.

Solver side:

  - 'SolverId' carries the build 'Stage' of the package it references,
    populated in 'convConfId' from the QPN. 'ResolverPackage.solverId'
    derives the stage-aware graph key ('nodeKey'), so host and build
    copies no longer collide in the solver plan.

Elaborated plan:

  - 'ElaboratedInstalledPackageInfo = WithStage InstalledPackageInfo',
    and 'ElaboratedConfiguredPackage''s node key becomes 'WithStage
    UnitId' ('nodeKey = WithStage elabStage elabUnitId'). The
    polymorphic-key install plan (haskell#12092) forces both node types to agree
    via 'Key ipkg ~ Key srcpkg'.
  - Every dependency edge carries the stage the solver actually resolved
    it at: library dependencies are on the package's own stage, while
    executable (build-tool) and setup dependencies are stored as
    'WithStage ConfiguredId' ('compExeDependencies', new
    'pkgSetupLibDependencies') with the stage of the resolved plan node.
    'nodeNeighbors' reads those stored stages. It deliberately does not
    recompute 'prevStage': in a non-cross build the solver keeps setup
    and tool dependencies on the host stage, so inventing a Build edge
    would dangle.
  - 'fromSolverInstallPlan{,WithProgress}', 'pruneInstallPlanPass1'
    (PrunedPackage) and 'instantiateInstallPlan' are all keyed by the
    stage-carrying key.

Boundaries: the build and monitoring subsystems stay keyed by plain
'UnitId' (BuildOutcomes, BuildStatusMap, plan.json, the various
commands). The stage tag is projected away there with
'withoutStage'/'installedUnitId'. In a non-cross build the projection is
lossless; under cross-compilation it is lossy whenever both copies share
a UnitId -- the host copy wins in the build maps, and plan.json lists
both under one id. Later parts of the series make those consumers
stage-aware.

No behaviour change for a non-cross build, which has a single stage.
Cross-compiling a package with a custom Setup now builds its setup
dependencies with the build compiler, as distinct plan nodes
(PackageTests/BuildCompilerSetup).
Pull the shared/profiling library-way selection out of
'elaborateInstallPlan' into a standalone stage-aware pass. Capability and
default-way decisions use each package's own stage compiler, and the
downward-closed way sets are keyed by (Stage, PackageId), so a host
package's way no longer leaks onto its build-stage twin.

Elaborating per stage left this selection pinned to the host compiler, so
until now a Build-stage package picked its library ways from the host
compiler's capabilities and defaults. This closes that.

No-op off cross-compilation.
…stage-aware

The solver's goal-linking (Builder.hs) and GHC's single-instance
restriction (Preference.hs) both grouped and keyed goals by package name
and instance, ignoring 'Stage' entirely. So the solver could "link"
(share a build of) a Setup/Build-stage goal with an unrelated Host-stage
goal of the same package and version, even though the two are compiled by
different toolchains and can never share a build artifact.

The easiest way to hit it is a local package that is also used as a build
tool: every local package is a solver goal at the host stage, and the
host package's build-tool-depends puts the same package on the build
stage. The two goals get linked, and once the build toolchain differs
from the host toolchain the solve fails with "dependencies not linked:
cannot merge" and "multiple instances" on their (different) 'base'.
PackageTests/BuildCompilerToolFromSource exercises exactly that. I first
hit it bootstrapping GHC with a two-stage build (ghc-9.8.4 as build
compiler, a freshly-built stage1 GHC as host): 'cabal build rts' failed
with spurious "cannot merge" and "multiple instances" errors on 'array',
'Cabal-syntax' and 'Cabal'.

  - Builder.hs: key 'LinkingState' by (PN, I, Stage) rather than (PN, I),
    so linking is only ever proposed between goals of the same stage.
  - Preference.hs: key 'enforceSingleInstanceRestriction''s tracking map
    by (Stage, PI PN) rather than (PI PN), so two *unlinked* goals of the
    same package and version are only forbidden within one stage. Across
    stages they land in different stores and do not collide.

Dependency.hs had a related but separate bug: the constraints derived
from the host compiler's wired-in units were stage-blind. The 'base >=
4.22' lower bound also applied to 'build:*:setup.base', and a Setup
component is compiled by the fixed boot compiler, which was never going
to satisfy that bound anyway; forcing it there pushed Setup's 'base' onto
local source, which pulled in 'ghc-internal', unsatisfiable for a boot
compiler that predates it. The exact-unit-id pins for the wired-in
packages carry the host compiler's unit ids for the same reason. Both are
now scoped to 'Just Host'. In a non-cross build every goal is on the host
stage, so nothing changes there.

The solver test DSL cannot express a second stage yet, so the new
behaviour is exercised by the cross e2e tests only.
The runner copies each test's source directory using 'git ls-files', so
from a source tree that is not a git checkout -- a jj workspace, an
unpacked sdist -- every test fails with "No files to copy". Fall back to
listing the directory, skipping the build artefacts git would have
ignored.

Also export 'ConfiguredInplace' from Test.Cabal.Plan, so tests can
inspect the in-place items of plan.json.
Earlier commits made every package select its own stage's toolchain for
finalisation and for the flags handed to an external Setup.hs, but the
in-process build path and the project-wide package-db plumbing stayed
pinned to the host toolchain. So under cross-compilation a build-stage
package built from source -- a build tool, a setup dependency -- was
configured, registered and looked up with the host compiler, in the host
package databases. This closes that.

  - ProjectPlanning: the per-package database stacks (build, register,
    setup, and their in-place variants) are derived from the package's
    own stage, meaning the store and dist databases of that stage's
    compiler on top of whichever project databases apply to the stage
    ('projectPackageDbsFor'), and the setup stacks from the previous
    stage. The host/build-specific top-level stacks are gone.
    'elabPackageDbs' is the package's own stage's list too, which keeps
    the register-stack assertion in ProjectBuilding honest and puts the
    databases a package was actually built against into its unit id
    hash.
  - ProjectBuilding: package databases are created and read per stage,
    with that stage's compiler and ghc-pkg -- a build compiler's db is
    not readable by the host's ghc-pkg. The running installed-package
    index is per stage too (a 'Staged' index, read and updated with the
    package's stage). The same package name is registered in both
    stages' databases (rts, base, ...), and a single merged index answers
    a lookup by name with units of the wrong stage: Cabal's hsc2hs
    preprocessor looks the rts up by name and dies on the ambiguity ("No
    (or multiple) ghc rts package is registered") when a build-stage
    package is configured in the library. A non-cross build has only the
    host index.
  - InLibrary / SetupWrapper / UnpackedPackage: the in-library setup
    method configures a package with the compiler, platform and program
    database of the package's stage
    ('elabCompiler'/'elabPlatform'/'elabProgramDb'), and the in-place
    registration uses the same.
  - solver: the validation phase checks extensions, languages and
    pkg-config requirements against the compiler and pkg-config database
    of the stage each goal is solved for, and
    'configuredPackageProblems' finalises a solved package with its own
    stage's compiler and platform, rather than the host's for everything.
  - The store side of an installed build-stage package: its store entry,
    package db and log file are its own stage's, and plan improvement
    reads the store of every active stage (unit ids hash the compiler, so
    the union of the entry sets is unambiguous).

Behaviour-preserving for a non-cross build: with a single stage every
selection resolves to the host toolchain as before.

Verified by the new PackageTests/BuildCompilerToolFromSource, which
builds a build-tool dependency and its library dependency from source
with a distinct build compiler, and checks which compiler each executable
and each copy of the library was compiled with. It needs the stage-aware
goal linking of the previous commit: without it the host-stage local copy
of the tool and its build-stage copy are linked and the solve fails with
"multiple instances".
Every entry of the install plan in plan.json gains a "stage" field,
"host" or "build". The plan is keyed by (stage, unit id), so under cross-compilation the same unit id can legitimately appear
twice -- the host copy of a package, and the build-stage copy used as a
build tool or setup dependency -- and a plan.json consumer had no way to
tell the two apart. In an ordinary build every entry is on the host
stage. Documented in nix-local-build.rst.

The test harness reads the field, defaulting to host for older files, and
'planDistDir' prefers the host-stage copy when a component is present at
both stages, which is what a test that ran "cabal build all" means by the
name. BuildCompilerToolFromSource selects the build-stage copy of its
tool through the field instead of guessing from the dist directory.

cabal repl: the two lookups of a target unit in the plan pick the
host-stage node explicitly ('InstallPlan.lookup' with a host-stage key)
rather than the first node with a matching unit id, which under
cross-compilation could be the build-stage copy.

Still projected onto bare UnitId at this point: BuildStatusMap and
BuildOutcomes, where the host copy wins and the next commit keys them by
stage, and the 'availableTargets'/'setRootTargets' keys, left for the build-dir
work, which is why a package used as a build tool becomes a root at both
stages under "all".
'BuildStatusMap' and 'BuildOutcomes' were keyed by 'UnitId' while the
install plan is keyed by 'WithStage UnitId'. The build phase projected
the plan keys down to unit ids at its boundaries -- a no-op in a
non-cross build -- and the consumers looked packages up by
'installedUnitId'.

Under cross-compilation a package can be in the plan at both stages, and
the two copies of an in-place package share their unit id:
"foo-1-inplace" is not compiler-specific. The projection then kept a
single status for both copies, last writer wins. Improving the plan with
the up-to-date packages read the other copy's status, so a package whose
own copy had never been built was marked installed while its
dependencies were still configured, which trips the invariant of
'InstallPlan.installed'. Building GHC's stage2 libraries with cabal hit
exactly this, on the build-stage copy of a local build tool.

Key both maps by the plan key instead, and drop the projections:

  - 'rebuildTargetsDryRun', 'rebuildTargets' and the download scan use
    'nodeKey';
  - the up-to-date improvement, the plan printer, build reports and the
    failure reporting look up by 'nodeKey', which also removes the
    "match on the projected unit id" workarounds in
    'dieOnBuildFailures';
  - the post-build project status ('PostBuildProjectStatus' and the
    persisted up-to-date set) is tracked per plan key, with library
    dependency edges on the package's own stage. An old up-to-date cache
    no longer decodes and is treated as empty, as any stale cache is;
  - the GHC environment file lists host-stage libraries only: a local
    package used as a build tool also has a build-stage copy, whose
    libraries are for the build compiler.

What remains keyed by bare 'UnitId' is the target selection side
('availableTargets', 'setRootTargets', 'TargetsMap'), which the build-dir
work will revisit.
…kage

A package that arrives as a tarball (Hackage, a remote tarball, a
source-repository-package) is unpacked once, under dist-newstyle/src, and
built from there. Under cross-compilation the build-stage and host-stage
copies of such a package are separate plan nodes, and with -j they are
scheduled concurrently: both find the source directory missing, both
unpack into it, and whichever loses the race sees a half-written tree and
fails -- "No cabal file found", or a truncated module.

Make the check-and-unpack a critical section on a lock shared by all
builds, as registration and the setup exe cache already are. Unpacking is
a small part of a build, so serialising it costs nothing measurable, and
the lock is only ever contended by the two stages of one package.
The existing BuildCompiler tests pass a genuinely different build
compiler, so a package appearing at both stages gets two distinct
UnitIds. When the two stages share a compiler the UnitIds are identical
instead, and anything keying the plan by bare UnitId conflates the
copies. That is the configuration GHC's own staged build uses -- its
stage1 passes the bootstrap compiler as both --with-compiler and
--with-build-compiler -- and nothing covered it.

BuildCompilerSameCompiler has mylib as a library dependency of the
host-stage app and of tool, which app depends on as a build tool, and
passes the host compiler as --with-build-compiler. It asserts from
plan.json that mylib and tool are each planned exactly once (since
'configureToolchains' recognises the two toolchains as the same compiler,
there is no separate build stage), and it runs the resulting executable,
because a bare 'cabal build' exits 0 even when it has not built what was
asked for. It needs no second GHC, so it always runs. Test.Cabal.Plan now
exports 'ConfiguredInplace' so tests can inspect local plan items.
'StoreDirLayout' keyed every location on the 'Compiler' alone:

    store/<compiler-id>-<abi>/<unit-id>

The compiler's id and ABI tag record which compiler built a package, but
not what that compiler produces. With a build stage and a host stage in
one build the two compilers can agree on both while targeting different
architectures, and their outputs are then filed in one directory, under
one package database. The 'UnitId's differ -- cabal hashes the platform
along with the compiler id, see 'PackageHashInputs' -- so the two builds
do not overwrite each other. What they share is the directory, left
holding binaries for two architectures, and the package database, left
holding entries for both and read back by two different ghc-pkgs.

Take the target 'Platform' alongside the 'Compiler' in every
'StoreDirLayout' location, and name it first in the path:

    store/<platform>/<compiler-id>-<abi>/<unit-id>

which is the segment order 'distBuildDirectory' has always used. Each
caller passes the platform belonging to the package at hand: the stage's
toolchain for a per-stage database or store entry, 'pkgConfigPlatform'
for the project as a whole, and 'configureCompiler''s platform in 'cabal
path'.

'distPackageDB', the database inplace packages are registered in, takes
the platform too:

    dist-newstyle/packagedb/<platform>/<compiler-id>

and for a sharper reason than the store. An inplace 'UnitId' does not
mention the compiler at all, so where the store's two copies merely
shared a directory, the two copies of a local package built at both
stages are registered over one another.

The test harness learns the same layout. 'findDependencyInStore' and the
'--intree-cabal-lib' package-db lookup both descend the extra segment,
and both tolerate a store without it: those libraries are built by the
cabal on PATH rather than the one under test, so they may still be
written in the old layout.

Existing store entries under the old layout are neither migrated nor
read. They are left in place, and packages are rebuilt into the new
layout on first use. The same goes for the inplace package database.
…igShared

'elaborateInstallPlan' called its 'ProjectConfigShared' argument
'sharedPackageConfig', which is the name ProjectBuilding.hs uses for an
'ElaboratedSharedConfig' -- a different type. So the same identifier meant
two things depending on the module.

Call it 'projectConfigShared', as the field and every other binding of
that config are named, which leaves 'sharedPackageConfig' to mean the
elaborated config and nothing else.

Rename only.
@andreabedini

Copy link
Copy Markdown
Collaborator Author

I'm going to remove myself from the reviewers here, because I'm under the impression that you've used LLMs to some extent, and I do not review non-trivial patches created with the help of LLMs in my spare time.

I have indeed being using Claude for my work for a while. It's fair if you prefer to spend your free time otherwise. Let me just point out that I have invested a lot of effort in this.

-- unconfigured attendant programs such as @hsc2hs@, @haddock@ and toolchain
-- programs such as @ar@, @ld@. See 'Distribution.Simple.GHC.configure'.
configureCompiler
configureToolchains

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the GHC configuration hardcodes environment variables, what do you think about how it will behave after configuration?
I have a suggestion:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented on the issue and on the PR. Generally speaking, I have been careful to thread the properties of the two compilers separately: separate programdb, separate packagedb, etc. So I don't make any other assumptions other than "there are two of them".

@zlonast

zlonast commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

I hope we add the host: syntax so we can run cabal install all and get all the necessary binaries at once.

@zlonast

zlonast commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Perhaps we should use --with-compiler after all and move the configuration to the cabal.project file? I might have misread something, but it seems like the flags for --build-host and --target-host would better reflect what we want to achieve?

@andreabedini

Copy link
Copy Markdown
Collaborator Author

I hope we add the host: syntax so we can run cabal install all and get all the necessary binaries at once.

cabal install all works just like before. In a cross-compilation setting it will install all the host executables. The host:/build: syntax would be a way to specify a stage for a target like cabal build build:happy.

@andreabedini

Copy link
Copy Markdown
Collaborator Author

Perhaps we should use --with-compiler after all and move the configuration to the cabal.project file? I might have misread something, but it seems like the flags for --build-host and --target-host would better reflect what we want to achieve?

The command line flags and project options already go in parallel. --with-compiler becomes compiler:, --with-build-compiler becomes build-compiler: etc.

W.r.t. --build-host/--target-host it is much easier (at least for now) to specify a compiler by name or path. Otherwise we will have to know how to tell the compiler which architecture to target. Also the autoconf target architecture only matters when building compilers, the other two are host and build, which match the terms I used in the PR.

@andreabedini
andreabedini force-pushed the andrea/feature/cross-compilation-2 branch from eaf4a1c to 3799e7f Compare September 17, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants