feat(node): add -slim image variants without preinstalled SDKs - #304
feat(node): add -slim image variants without preinstalled SDKs#304vladfrangu wants to merge 6 commits into
Conversation
|
So how can we release this without crashing the world (since the template Dockerfile itself crashes)? Will it be forever opt-in? We introduce it from Node:26 or later? |
|
People should NOT be depending on pre-installed modules. I personally think this breaking change is fine, but it's going to be separate so we can revert it quickly if fires light up, and I can then turn this into an opt-in slim build |
|
We could introduce new image tags (e.g. add I also feel like this should be fine and people don't really depend on having stuff preinstalled, but having a different set of image tags would surely be safer. I would rather not bind this to node versions, it'd be pretty confusing. |
These things don't go together. Every template user from past X years would crash on rebuild, it cannot be much more breaking than that. Agree that relying on preinstalled packages alone would be okish to break. |
|
GPT offers some hacks to fake the npm command but that might open another can of worms. |
|
I would rather make a tagged slim image than a hacky npm wrapper 😅 |
|
Let's just introduce new tags as I mentioned above, no need for some weird hacks.
This shouldn't fail, running |
|
sooo, whats the deal? Is there something we can help you guys with? |
It does fail npm ls nonexistent-package |
|
It only fails if all packages are missing. We already check for both playwright and puppeteer this way, and we almost never have both around. |
0d52600 to
d3003b6
Compare
d955df7 to
1637bd8
Compare
Every node image release now also builds and pushes a -slim variant of each tag (e.g. apify/actor-node:24-slim, apify/actor-node-playwright-chrome:24-1.60.0-slim). Each image dir gains a package.slim.json — the same manifest minus apify, crawlee and typescript (and the crawlee overrides), keeping only the browser library the image is built around. Slim images are built from the same Dockerfiles with SLIM=1, which installs from package.slim.json instead of package.json; regular tags are unchanged. set-dependency-versions.js keeps both manifests' versions in sync during releases. Actors on -slim get exactly the dependency versions in their own package.json instead of whatever the image preinstalled, at the cost of installing them at build time. The default main.js smoke tests no longer depend on apify or crawlee (which -slim images don't ship): apify is imported optionally, and the browsers are driven with raw playwright/puppeteer instead of crawlee's launchers, so one test suite serves both variants.
1637bd8 to
2777e63
Compare
The optional require('apify') caught every error, so a regular image with a
corrupted apify install would silently degrade to the slim test path and pass.
Only tolerate apify itself being absent (the -slim case); rethrow anything
else. When apify is present, also require crawlee, so the regular images
verify both preinstalled SDKs again.
… test chrome_test.js only launched full Chrome (channel: 'chrome'), so the bundled Chromium the image installs into PLAYWRIGHT_BROWSERS_PATH - and the APIFY_DEFAULT_BROWSER_PATH symlink pointing at it - were never exercised. Launch it both ways before the Chrome tests.
- Run `execSync('ps')` in every image smoke test: official Node builds have
occasionally shipped without ps, which crawlee needs at runtime to measure
memory. The old getMemoryInfo() call guarded this; restore it without a
crawlee dependency so it also covers the -slim images.
- Fix the copy-pasted SLIM comment in node/Dockerfile (the base image has no
browser library).
- Add the missing blank line between the regular and slim push steps in the
release workflows.
Rethrow non-ENOENT errors when reading package manifests so a malformed file fails loudly, drop the unused testPageLoading export in the chrome image test, and correct the node-playwright manifest descriptions.
Adds the -slim tag family to the README, makes update-package-json.mjs update package.slim.json alongside package.json, and adds a SLIM=1 flag to the node Makefile targets (passed as a build arg, with the slim manifest restored after the build).
B4nan
left a comment
There was a problem hiding this comment.
added some fixes, please review those before merging
| } else { | ||
| run().catch((error) => { | ||
| console.error(error); | ||
| process.exitCode = 1; |
|
Should we revive the non-root user (#197) for the slim images? The time is now, I don't think we'll get such an opportunity anytime soon. |
What
Every node image release now also builds and pushes a
-slimvariant of each tag, alongside the unchanged regular images:Slim images are built 1:1 from the same Dockerfiles with a new
SLIM=1build arg. Each image dir gains apackage.slim.json— the same manifest minusapify,crawleeandtypescript(and the crawleeoverrides); the slim build installs from it instead ofpackage.json. Each image keeps only the browser library it is built around:-slimstill preinstallsnodenode-playwrightplaywrightnode-playwright-chromeplaywright-chromiumnode-playwright-firefoxplaywright-firefoxnode-playwright-webkitplaywright-webkitnode-playwright-camoufoxcamoufox-js,playwright,impitnode-puppeteer-chromepuppeteerHow
<image>/package.slim.json(all 7): slim manifest, reviewed as a plain file — no JSON munging at build time.ARG SLIM=0selects one:set-dependency-versions.js: now substitutes release versions in bothpackage.jsonandpackage.slim.json, so the manifests can't drift on release.prepare-node-image-tags.js: optionalTAG_SUFFIXenv appended to every generated tag.TAG_SUFFIX: "-slim"), Build and tag slim image for testing (SLIM=1), Test slim image, and Build and push slim OCI image. Slim builds use a-slimGHA cache scope and fall back to the regular scope for the shared layer prefix.main.js+ helpers): no longer depend on apify/crawlee (which-slimdoesn't ship) —apifyis imported optionally and browsers are driven with raw playwright/puppeteer, so one test suite serves both variants.Why
Preinstalled SDK versions go stale immediately and rarely match what an Actor's own
package.jsonpins — most builds install their own copies anyway, on top of pulling image layers containing the unused preinstalled ones. The-slimvariants give Actors exactly the dependency versions they declare, with much smaller pulls, without breaking anyone on the existing tags.Verification
Local builds of
actor-node:24from this branch:apify,crawlee,typescript(+overrides)SLIM=1Both variants' self-tests pass (
... test PASSED), andpackage.slim.jsonis confirmed absent from both final images.set-dependency-versions.jsverified to substitute versions in both manifests (checked withnode-puppeteer-chrome+PUPPETEER_VERSION). Tag generation unit-checked:latest/20→latest-slim/20-slim;beta+ framework1.60.0→beta-slim,20-1.60.0-beta-slim,20-beta-slim. All three workflow files parse as valid YAML. The slim browser-image test content matches what already passed CI in theempty-cached/empty-uncachedexperimental builds.