feat(pgpm): init adds each new module to the workspace CI matrix, sorted by pyramation · Pull Request #1845 · constructive-io/constructive

pyramation · 2026-09-17T07:36:36Z

Summary

The generated workspace CI keeps a literal, hand-editable test matrix; pgpm init (module) now maintains it instead of the matrix being computed at CI time with pgpm ls (the "magic" objected to in pgpm-boilerplates#53/#55).

After a module is scaffolded (pgpm and pnpm/lerna/npm workspaces), init calls the new addToCiMatrix(workspacePath, relativeModulePath) from @pgpmjs/core:

scans <workspace>/.github/workflows/*.y(a)ml
parses each file with the yaml package (new dep on @pgpmjs/core) and resolves jobs.<job>.strategy.matrix.package for every job — a package: key anywhere else (a step's with:, env:) is never a match
for each such YAML sequence of plain strings: inserts the module's workspace-relative path (packages/<name>), dedupes, sorts with localeCompare, and splices the re-rendered sequence into its own range in the source; the rest of the file is passed through byte-for-byte, so comments, quoting and indentation survive (no stringify round-trip)
keeps the sequence's style — flow [a, b] (single- or multi-line) or block - a — existing items' source text as written, and each item's own comments
no-op when the entry is already present, the file has no such matrix, the matrix isn't a plain string list (${{ fromJSON(...) }}), or the YAML doesn't parse
prints Added <mod> to the CI matrix in .github/workflows/ci.yml per changed file
best-effort: any fs error (unreadable/unwritable workflow, .github/workflows being a file) skips that file/dir and never fails pgpm init — the module is already scaffolded by then, so a CI-file problem must not blow up the command

# before                         # after `pgpm init` → alpha
package: []                      package: [packages/alpha]
package:                         package:
  - packages/beta                  - packages/alpha
  - packages/delta                 - packages/beta
                                   - packages/delta

Hand-edited arrays keep their comments

A user's matrix is theirs to edit, so per-item comments travel with their item through the sort rather than being dropped:

package:                          package:
  - packages/beta   # api    →      - packages/alpha
  # the web app                     - packages/beta   # api
  - packages/delta                  # the web app
                                    - packages/delta

commentBefore/comment are read off each scalar and re-emitted with it; when the first item carries a leading comment block the replaced span is extended back over those lines so they can't be duplicated. A flow sequence containing comments is left untouched instead (re-rendering comments inside [...] isn't worth it).

Backstop, so no shape can silently eat a comment: before an edit is accepted, the candidate result is re-parsed and its comment multiset compared with the original document's (commentsOf, via yaml's visit). Any loss, duplication, or parse failure → that matrix is left exactly as it was.

Companion boilerplate change (ships package: [] + cd ./${{ matrix.package }}): constructive-io/pgpm-boilerplates#56. Note GitHub rejects an empty literal matrix, so a workspace with zero modules has an invalid ci.yml until its first pgpm init; pgpm init -w produces a valid one immediately.

Tests: pgpm/core/__tests__/core/ci-matrix.test.ts — whole-file equality for flow/block/empty/quoted/multi-job/unrelated-key/expression/unparseable/CRLF/anchor, plus a table-driven case asserting the comment multiset is identical before and after across 25 hand-edited shapes (extra spaces, trailing comments, comment blocks above items, multi-line flow, # inside quoted values). E2E in pgpm/cli/__tests__/init.test.ts inits zeta then alpha and asserts package: [packages/alpha, packages/zeta] with the surrounding comment preserved.

Link to Devin session: https://app.devin.ai/sessions/dd4055c50bcf4b1e8e604211038aaa82
Open in Devin Desktop: https://app.devin.ai/desktop/session/dd4055c50bcf4b1e8e604211038aaa82?variant=devin
Requested by: @pyramation

devin-ai-integration · 2026-09-17T07:36:39Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

pyramation · 2026-09-17T08:00:15Z

+ *
+ * Returns the workflow files that changed, relative to `workspacePath`.
+ */
+export const addToCiMatrix = (


explain how this works. explain also how we read/write yaml and ensure precision.

How it works

addToCiMatrix(workspacePath, 'packages/<mod>') runs once, right after pgpm init has scaffolded a module. It lists <workspace>/.github/workflows/*.y(a)ml and hands each file to addToMatrixYaml(source, entry), which is a pure string -> string function. It writes a file back only if the string changed, and returns the list of files it touched (init prints one line per file).

addToMatrixYaml walks the file line by line looking for the first package: key and handles the two ways a YAML sequence can be written:

flow — ^(\s*)package:\s*\[(.*)\]\s*$ → split the bracket contents on ,, strip quotes, add the entry, dedupe, sort, and rewrite that one line as <same indent>package: [a, b, c].

block — ^(\s*)package:\s*$ → collect the following - item lines that are indented deeper than the key and all at the same indent; add/dedupe/sort; splice the new - item lines into exactly that range, reusing the original item indent.

Everything else in the file — comments, blank lines, other jobs, ${{ }} expressions, anchors — is never parsed, so it's never touched. Sorting is localeCompare; dedupe is a Set, so re-running init for an existing module is a no-op.

Why not js-yaml

We deliberately do not yaml.load() → mutate → yaml.dump(). js-yaml (and yaml round-trippers generally) throw away comments, re-quote strings, reflow >-/| blocks and re-indent the whole document — which would rewrite a file the user is supposed to own and hand-edit. Precision comes from only ever replacing the line(s) that make up the package: sequence; the rest of the buffer is passed through byte-for-byte (we split on \n and join on \n, so line endings are preserved too).

Guardrails / when it backs off (returns the input unchanged)

no .github/workflows/ dir, or no file contains a package: list

package: followed by something that isn't a sequence (a mapping, ${{ fromJSON(...) }}, a scalar) — we only rewrite a sequence we fully parsed

entry already present

Trade-off we accept: the matrix key is the literal name package (what the templates ship), and a block list ends at the first line that isn't a - item at that indent — so a comment between items would truncate the recognized range. Both are covered in ci-matrix.test.ts (flow/block/empty/quoted/no-op/non-sequence) plus an e2e in init.test.ts that inits zeta then alpha and asserts package: [packages/alpha, packages/zeta] with the comment above it intact.

Superseded by ef214c9 — the hand-rolled line parser is gone.

Read: parseDocument(source) from the yaml package (already in the repo's dep tree; added to @pgpmjs/core). Any parse error → the file is returned unchanged. The matrix is located structurally: for each key under jobs, doc.getIn(['jobs', job, 'strategy', 'matrix', 'package'], true); it has to be a YAMLSeq whose items are all string scalars.

Write: we don't stringify the document (that would reflow comments, quoting and indentation across the whole workflow). Every yaml node carries range: [start, end] byte offsets into the original source, so we take the existing items' source text (quotes preserved as written), add the new entry (stringify(entry) so it's quoted only if YAML needs it), dedupe, sort with localeCompare, re-render the sequence in its original style (flow [a, b] or block - a, at the original indent), and splice that into the sequence's own [start, end] range. Multiple matrices are spliced last-to-first so offsets stay valid. Every other byte of the file is passed through untouched.

Tests in pgpm/core/__tests__/core/ci-matrix.test.ts assert full-file equality (not toContain) for flow, block, empty [], quoted entries, multi-job, unrelated package keys, ${{ }} matrices and unparseable YAML.

pyramation · 2026-09-17T08:00:55Z

+
+  for (let index = 0; index < lines.length; index += 1) {
+    const flow = lines[index].match(
+      new RegExp(`^(\\s*)${MATRIX_KEY}:\\s*\\[(.*)\\]\\s*$`)


how do we know it's the matrix key and not some other key for another workflow?

Before ef214c9 we didn't — that regex matched any package: line, including one under a step's with: or env:. Fixed: the file is now parsed with the yaml package and the updater resolves jobs.<job>.strategy.matrix.package for each job via doc.getIn([...]). Only a YAML sequence at exactly that path is treated as a matrix; a package key anywhere else is ignored, and so is a matrix that isn't a plain list of strings (e.g. ${{ fromJSON(...) }}). Tests: "ignores a package key that is not a job matrix", "updates the matrix of every job that has one", "leaves a matrix that is not a plain list of strings alone".

…jobs.*.strategy.matrix.package

feat(pgpm): init adds each new module to the workspace CI matrix, sorted

2fad719

devin-ai-integration Bot assigned pyramation Sep 17, 2026

pyramation mentioned this pull request Sep 17, 2026

ci: ship an empty literal test matrix that pgpm init fills in constructive-io/pgpm-boilerplates#56

Merged

pyramation commented Sep 17, 2026

View reviewed changes

pyramation added 5 commits September 17, 2026 08:05

refactor(pgpm): locate the CI matrix with the yaml parser, scoped to …

ef214c9

…jobs.*.strategy.matrix.package

Merge origin/main into feat/pgpm-init-ci-matrix

ee7256d

fix(pgpm): preserve CI matrix comments and flow formatting

2730821

fix(pgpm): validate preserved CI matrix comments

8bca250

fix(pgpm): never let CI matrix update fail init

689657b

pyramation merged commit 472842b into main Sep 17, 2026
20 checks passed

pyramation deleted the feat/pgpm-init-ci-matrix branch September 17, 2026 22:59

Navigation Menu

feat(pgpm): init adds each new module to the workspace CI matrix, sorted - #1845

feat(pgpm): init adds each new module to the workspace CI matrix, sorted#1845
pyramation merged 6 commits into
mainfrom
feat/pgpm-init-ci-matrix

pyramation commented Sep 17, 2026 •

edited by devin-ai-integration Bot

Loading

Uh oh!

devin-ai-integration Bot commented Sep 17, 2026

Uh oh!

pyramation Sep 17, 2026

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Uh oh!

pyramation Sep 17, 2026

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

pyramation commented Sep 17, 2026 • edited by devin-ai-integration Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Hand-edited arrays keep their comments

Uh oh!

devin-ai-integration Bot commented Sep 17, 2026

🤖 Devin AI Engineer

Uh oh!

pyramation Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

pyramation Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

pyramation commented Sep 17, 2026 •

edited by devin-ai-integration Bot

Loading