Skip to content

feat(node): add -slim image variants without preinstalled SDKs - #304

Open
vladfrangu wants to merge 6 commits into
masterfrom
feat/no-preinstalled-deps
Open

feat(node): add -slim image variants without preinstalled SDKs#304
vladfrangu wants to merge 6 commits into
masterfrom
feat/no-preinstalled-deps

Conversation

@vladfrangu

@vladfrangu vladfrangu commented Aug 4, 2026

Copy link
Copy Markdown
Member

What

Every node image release now also builds and pushes a -slim variant of each tag, alongside the unchanged regular images:

apify/actor-node:24            →  apify/actor-node:24-slim
apify/actor-node:latest        →  apify/actor-node:latest-slim
apify/actor-node-playwright-chrome:24-1.60.0  →  apify/actor-node-playwright-chrome:24-1.60.0-slim
apify/actor-node:24-beta       →  apify/actor-node:24-beta-slim

Slim images are built 1:1 from the same Dockerfiles with a new SLIM=1 build arg. Each image dir gains a package.slim.json — the same manifest minus apify, crawlee and typescript (and the crawlee overrides); the slim build installs from it instead of package.json. Each image keeps only the browser library it is built around:

Image -slim still preinstalls
node (nothing)
node-playwright playwright
node-playwright-chrome playwright-chromium
node-playwright-firefox playwright-firefox
node-playwright-webkit playwright-webkit
node-playwright-camoufox camoufox-js, playwright, impit
node-puppeteer-chrome puppeteer

How

  • <image>/package.slim.json (all 7): slim manifest, reviewed as a plain file — no JSON munging at build time.
  • Dockerfiles (all 7): both manifests are copied in; ARG SLIM=0 selects one:
    RUN if [ "$SLIM" = "1" ]; then mv package.slim.json package.json; else rm package.slim.json; fi
    Regular builds are unchanged in behavior, and neither variant ships the extra file.
  • set-dependency-versions.js: now substitutes release versions in both package.json and package.slim.json, so the manifests can't drift on release.
  • prepare-node-image-tags.js: optional TAG_SUFFIX env appended to every generated tag.
  • Release workflows (node / playwright / puppeteer): each build job gains mirrored steps — Prepare slim image tags (TAG_SUFFIX: "-slim"), Build and tag slim image for testing (SLIM=1), Test slim image, and Build and push slim OCI image. Slim builds use a -slim GHA cache scope and fall back to the regular scope for the shared layer prefix.
  • Smoke tests (main.js + helpers): no longer depend on apify/crawlee (which -slim doesn't ship) — apify is imported optionally and browsers are driven with raw playwright/puppeteer, so one test suite serves both variants.

Why

Preinstalled SDK versions go stale immediately and rarely match what an Actor's own package.json pins — most builds install their own copies anyway, on top of pulling image layers containing the unused preinstalled ones. The -slim variants give Actors exactly the dependency versions they declare, with much smaller pulls, without breaking anyone on the existing tags.

Verification

Local builds of actor-node:24 from this branch:

Variant Deps in image Size
regular apify, crawlee, typescript (+overrides) 497 MB
SLIM=1 (none) 164 MB

Both variants' self-tests pass (... test PASSED), and package.slim.json is confirmed absent from both final images. set-dependency-versions.js verified to substitute versions in both manifests (checked with node-puppeteer-chrome + PUPPETEER_VERSION). Tag generation unit-checked: latest/20latest-slim/20-slim; beta + framework 1.60.0beta-slim, 20-1.60.0-beta-slim, 20-beta-slim. All three workflow files parse as valid YAML. The slim browser-image test content matches what already passed CI in the empty-cached/empty-uncached experimental builds.

@github-actions github-actions Bot added this to the 146th sprint - Tooling team milestone Aug 4, 2026
@github-actions github-actions Bot added t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics. labels Aug 4, 2026
@vladfrangu vladfrangu added the adhoc Ad-hoc unplanned task added during the sprint. label Aug 4, 2026
@JuanGalilea

JuanGalilea commented Aug 10, 2026

Copy link
Copy Markdown

broader version of #289

could we get a sneaky review @B4nan 🙏 😅

Image size gains are too good to pass up! 👀
We got ~200mb final image size decrease with these changes (which is like ~40% of the total size)

@metalwarrior665

Copy link
Copy Markdown
Member

So how can we release this without crashing the world (since the template Dockerfile itself crashes)? Will it be forever opt-in? We introduce it from Node:26 or later?

@vladfrangu

Copy link
Copy Markdown
Member Author

People should NOT be depending on pre-installed modules. I personally think this breaking change is fine, but it's going to be separate so we can revert it quickly if fires light up, and I can then turn this into an opt-in slim build

@B4nan

B4nan commented Aug 10, 2026

Copy link
Copy Markdown
Member

We could introduce new image tags (e.g. add v2 or similar) and let the old ones work the same (we'd stop maintaining those).

I also feel like this should be fine and people don't really depend on having stuff preinstalled, but having a different set of image tags would surely be safer.

I would rather not bind this to node versions, it'd be pretty confusing.

@metalwarrior665

metalwarrior665 commented Aug 10, 2026

Copy link
Copy Markdown
Member

I personally think this breaking change is fine

Actor templates that run RUN npm ls @crawlee/core apify puppeteer playwright as a preinstall sanity check will fail on these images

These things don't go together. Every template user from past X years would crash on rebuild, it cannot be much more breaking than that. Agree that relying on preinstalled packages alone would be okish to break.

@metalwarrior665

Copy link
Copy Markdown
Member

GPT offers some hacks to fake the npm command but that might open another can of worms.

@vladfrangu

Copy link
Copy Markdown
Member Author

I would rather make a tagged slim image than a hacky npm wrapper 😅

@B4nan

B4nan commented Aug 11, 2026

Copy link
Copy Markdown
Member

Let's just introduce new tags as I mentioned above, no need for some weird hacks.

Actor templates that run RUN npm ls @crawlee/core apify puppeteer playwright as a preinstall sanity check will fail on these images

This shouldn't fail, running npm ls on a package that is missing doesn't fail, it just doesn't print the package that is not installed.

@JuanGalilea

Copy link
Copy Markdown

sooo, whats the deal?

Is there something we can help you guys with?

@metalwarrior665

Copy link
Copy Markdown
Member

This shouldn't fail, running npm ls on a package that is missing doesn't fail, it just doesn't print the package that is not installed.

It does fail

npm ls nonexistent-package
echo $? (prints 1)

@B4nan

B4nan commented Aug 18, 2026

Copy link
Copy Markdown
Member

It only fails if all packages are missing. We already check for both playwright and puppeteer this way, and we almost never have both around.

@vladfrangu
vladfrangu force-pushed the feat/no-preinstalled-deps branch from 0d52600 to d3003b6 Compare August 18, 2026 19:24
@vladfrangu vladfrangu changed the title feat(node)!: stop preinstalling apify, crawlee and typescript feat(node): add -slim image variants without preinstalled SDKs Aug 18, 2026
@vladfrangu
vladfrangu force-pushed the feat/no-preinstalled-deps branch 2 times, most recently from d955df7 to 1637bd8 Compare August 19, 2026 09:34
@vladfrangu
vladfrangu requested a review from B4nan August 20, 2026 12:04
Every node image release now also builds and pushes a -slim variant of each
tag (e.g. apify/actor-node:24-slim, apify/actor-node-playwright-chrome:24-1.60.0-slim).

Each image dir gains a package.slim.json — the same manifest minus apify,
crawlee and typescript (and the crawlee overrides), keeping only the browser
library the image is built around. Slim images are built from the same
Dockerfiles with SLIM=1, which installs from package.slim.json instead of
package.json; regular tags are unchanged. set-dependency-versions.js keeps
both manifests' versions in sync during releases.

Actors on -slim get exactly the dependency versions in their own package.json
instead of whatever the image preinstalled, at the cost of installing them at
build time.

The default main.js smoke tests no longer depend on apify or crawlee (which
-slim images don't ship): apify is imported optionally, and the browsers are
driven with raw playwright/puppeteer instead of crawlee's launchers, so one
test suite serves both variants.
@vladfrangu
vladfrangu force-pushed the feat/no-preinstalled-deps branch from 1637bd8 to 2777e63 Compare August 20, 2026 12:22
vladfrangu and others added 5 commits August 20, 2026 15:38
The optional require('apify') caught every error, so a regular image with a
corrupted apify install would silently degrade to the slim test path and pass.
Only tolerate apify itself being absent (the -slim case); rethrow anything
else. When apify is present, also require crawlee, so the regular images
verify both preinstalled SDKs again.
… test

chrome_test.js only launched full Chrome (channel: 'chrome'), so the bundled
Chromium the image installs into PLAYWRIGHT_BROWSERS_PATH - and the
APIFY_DEFAULT_BROWSER_PATH symlink pointing at it - were never exercised.
Launch it both ways before the Chrome tests.
- Run `execSync('ps')` in every image smoke test: official Node builds have
  occasionally shipped without ps, which crawlee needs at runtime to measure
  memory. The old getMemoryInfo() call guarded this; restore it without a
  crawlee dependency so it also covers the -slim images.
- Fix the copy-pasted SLIM comment in node/Dockerfile (the base image has no
  browser library).
- Add the missing blank line between the regular and slim push steps in the
  release workflows.
Rethrow non-ENOENT errors when reading package manifests so a malformed
file fails loudly, drop the unused testPageLoading export in the chrome
image test, and correct the node-playwright manifest descriptions.
Adds the -slim tag family to the README, makes update-package-json.mjs
update package.slim.json alongside package.json, and adds a SLIM=1 flag
to the node Makefile targets (passed as a build arg, with the slim
manifest restored after the build).

@B4nan B4nan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added some fixes, please review those before merging

} else {
run().catch((error) => {
console.error(error);
process.exitCode = 1;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this over process.exit(1)?

@B4nan

B4nan commented Aug 20, 2026

Copy link
Copy Markdown
Member

Should we revive the non-root user (#197) for the slim images? The time is now, I don't think we'll get such an opportunity anytime soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants