fix(docker): install the composer dependencies of cloned shipped apps

[luflow]

Note

Stacked on #1063 — the first commit belongs to that PR. Please merge #1063 first; without it configureNextcloud aborts before it ever reaches this code path when an app is bind mounted.

Fixes #1059. Companion to nextcloud/notifications#3206, where @nickvergessen pointed at this exact spot in lib/docker.ts as the place to solve it.

Problem

Shipped apps that are missing from the CI server image are installed with a plain git clone, which only works as long as the app commits its composer dependencies.

Since stable34, nextcloud/notifications no longer bundles them, but still ships a composer/autoload.php doing require_once __DIR__ . '/../vendor/autoload.php' — and vendor/ is git ignored. OC_App::registerAutoloading() finds that file, requires it, and every occ call dies:

Error: Failed opening required '/var/www/html/apps/notifications/composer/../vendor/autoload.php'
  … in /var/www/html/apps/notifications/composer/autoload.php:9
#0 /var/www/html/lib/private/legacy/OC_App.php(117): require_once()
#1 /var/www/html/lib/private/AppFramework/Bootstrap/Coordinator.php(78): OC_App::registerAutoloading()

On stable33 the composer/ directory did not exist yet, which is why this only surfaces when a suite moves to stable34. 0.4.0 and 0.5.0 are affected alike — the clone logic is unchanged between them.

Detection

Not a name list, and deliberately not the obvious heuristic either.

My first attempt was "composer/autoload.php exists but vendor/autoload.php does not". That is wrong: viewer and text commit a self-contained composer/ directory whose entrypoint loads composer/composer/autoload_real.php, not ../vendor/autoload.php. They are complete, and the heuristic triggered an unnecessary composer install for both (caught by npm run test:node).

So instead the app's autoloader is executed the same way the server does:

const { exitCode } = await runExec(['php', '-f', `${appPath}/composer/autoload.php`], { container, failOnError: false })

That detects exactly the apps that would break the server, with no assumption about how an app lays out its dependencies.

Chosen solution: composer install inside the container

The server image (ghcr.io/nextcloud/continuous-integration-shallow-server) ships PHP, git, curl, unzip and the zip/phar extensions, but no composer, so a pinned composer.phar is downloaded once per container.

Scripts are run on purpose: apps like notifications only assemble the prefixed copies of their dependencies (lib/Vendor/, via bamarni/composer-bin-plugin + coenjacobs/mozart in post-install-cmd) there. Skipping scripts would leave the app loadable but still broken for web push.

If the download or the install fails, the cloned composer directory is removed instead. The server then registers PSR-4 for the app's lib directory itself — the stable33 behaviour — so the app loads. It is incomplete, and that is logged loudly rather than passing silently:

│  ├─ notifications was cloned without the Composer dependencies its autoloader needs
│  ├─ 🛑 'composer install' failed for notifications: …
│  ├─ Removed 'notifications/composer' to make the app loadable without its Composer autoloader
│  └─ ⚠️  notifications is incomplete, code using its Composer dependencies (e.g. 'lib/Vendor') will fail

Apps that do bundle their dependencies are unaffected; the public API is unchanged.

Alternatives considered

• composer install on the host, result shipped in via container.putArchive (the workaround by @theCalcaholic in Missing bundled PHP dependencies (stable34) nextcloud/notifications#3206). Works, but adds a host dependency on PHP + composer. Given on ubuntu-latest, not given for contributors running the suite locally. The in-container variant behaves identically on Linux runners and on macOS with Docker Desktop.
• Only removing the cloned composer/ directory (my own workaround in Support installing composer packages for Nextcloud apps #1059). No toolchain needed at all, but lib/Vendor/ stays absent, so anything touching web push logs Class "OCA\Notifications\Vendor\Minishlink\WebPush\VAPID" not found. Kept as the degraded fallback, not as the default.
• Installing such apps from the app store instead of cloning — app store tarballs do ship vendor/, and it would remove this whole code path. Rejected because it gives up matching the app to the server branch, which is the reason the clone exists.
• Shipping composer in the CI server image. Arguably the right layer, and I would happily drop the download if that lands in continuous-integration-shallow-server. Until then this keeps consumers working without an image change.
• Adding an option to opt out of the install (for suites that only assert on in-app notifications and want the setup to stay fast). Left out on purpose — no caller needs it yet, and "skip the install" really means "install the app in a knowingly incomplete state", which is not a nice API to commit to. Happy to add it if you want one.
• Verifying the downloaded composer.phar against its published checksum. Dropped: it is served from the same host over the same HTTPS connection, so it proves transfer integrity, not authenticity — while the code deliberately executes third-party scripts from a fresh git clone moments later. The composer version is pinned instead, so a bad composer release cannot break every consumer's CI at once.

Testing

Fork checked out at this branch:

• npm ci, npx tsc --noEmit, npm run lint, npm run build — clean (lint: 0 errors; the 8 warnings are pre-existing in lib/commands/*)
• npm run test:node — 4/4 pass (47s). notifications added to the suite plus a new assertion that its vendor/autoload.php exists and the app is enabled.
• Non-regression: the run log shows the probe running for viewer/text/forms and not triggering an install — they keep the exact previous behaviour.

End-to-end against a real stable34 server, using nextcloud/attendance (the consumer from #1059) with this build linked in and its test server bumped stable33 → stable34:

├─ stderr: Cloning into 'apps-writable/notifications'...
│  ├─ notifications was cloned without the Composer dependencies its autoloader needs
│  ├─ Downloading Composer 2.10.2 into the container…
│  ├─ Running 'composer install' for notifications…
│  └─ Composer dependencies of notifications installed
├─ stdout: notifications 7.0.0-dev.1 enabled
├─ stdout: attendance 1.41.2 enabled
└─ Nextcloud is now ready to use 🎉
✅ Snapshot "init" created successfully!

Completeness checked in the container, not just "it boots":

$ docker exec … ls /var/www/html/apps-writable/notifications/lib/Vendor | head -3
Base64Url
Brick
GuzzleHttp

$ docker exec -u www-data … php occ app:list | grep -E 'notifications|attendance'
  - attendance: 1.41.2
  - notifications: 7.0.0-dev.1

$ docker exec … sh -c "tail -c 4000 /var/www/html/data/nextcloud.log" | grep -ci vapid
0

The degraded path got verified by accident and is worth reporting: an intermediate revision passed an invalid --no-audit flag. The run then produced exactly the intended behaviour — loud 🛑, composer/ removed, notifications 8.0.0-dev.0 enabled — and npm run test:node went red, with occ app:list emitting Class "OCA\Notifications\Vendor\…". So the fallback works, is visible, and the new test catches it as a regression. The flag is gone; everything above is from the final revision.

Not verified

• Linux runners — everything above ran on macOS + Docker Desktop.
• The full Playwright suite of the consuming app against stable34 (only the server setup was exercised; test failures there would be NC33→NC34 UI changes, unrelated to this).
• Apps other than notifications hitting this — the detection is generic, but notifications is the only app I saw it trigger for.
• The composer download failure path specifically; the fallback was reached via a failing install. Same structure, but untested for the download.

[susnux]

Suggested change

	* Working directory to run the command in. Defaults to the container's working directory.
	* Working directory to run the command in. Defaults to the Nextcloud root.

[susnux]

(or similar - as otherwise you would need to know the container structure)

[luflow]

Applied, thanks. It's also more accurate than what I had, since runExec(['cat', 'core/shipped.json']) and runOcc already rely on that default.

[susnux]

Why not simply:
IF composer.json exists
THEN composer install --no-dev

?

[luflow]

That was my first attempt, but it breaks on the two different layouts these apps use.

viewer and text have a second composer project in composer/composer.json with "vendor-dir": ".", so composer/ is their vendor dir and it's committed (composer/composer/autoload_real.php is right there in the repo). Their root composer.json is just dev tooling: viewer has an empty require, text only requires php.

notifications is the newer layout. Root composer.json is the runtime project, the vendor dir is the default vendor and git ignored, and composer/autoload.php is a hand written shim that requires it.

So "root composer.json exists, run install" would install the dev tooling project for viewer and text, while their actual runtime autoloader in composer/ is already fine. Nothing gets fixed, it just costs time.

The bigger problem is the combination with the rm -rf composer fallback. If that pointless install fails for any reason, we'd delete a committed, working vendor dir and break an app that was healthy. Running the autoloader only touches apps that would really make the server fatal, and it costs two docker exec round trips per cloned app.

Happy to go your way if you still prefer it. I'd just keep the fallback behind the autoloader check so a working composer/ can never get removed.

[susnux]

I doubt this will get updated especially apps will then update this library.
I think the risk of a broken composer release (until its fixed) is lower than being pinned to a vulnerable version.
So I prefer using latest here (you might want to add an environment override option though).

[nickvergessen]

Or call self-update afterwards

[luflow]

Fair enough, the vulnerable version argument beats mine. Back to latest-stable with an override:

const COMPOSER_VERSION = process.env.NEXTCLOUD_E2E_COMPOSER_VERSION || 'latest-stable'

@nickvergessen with latest-stable self-update doesn't do anything, so I left it out. Happy to add it if you'd rather pin and update.

I avoided the name COMPOSER_VERSION because a few PHP and composer Docker images set that one already, and picking it up by accident would be confusing. Can rename if you like.