test: add StrykerJS mutation testing foundation by ToRyVand · Pull Request #847 · lnp2pBot/bot

ToRyVand · 2026-06-26T05:39:46Z

Part of #760. First, foundational PR: set up StrykerJS and establish a baseline on util/. Writing tests for surviving mutants and raising the threshold will follow as separate focused PRs.

What this adds

@stryker-mutator/core + mocha-runner + typescript-checker
stryker.config.json — mocha runner, perTest coverage, non-blocking (break: null), scoped to util/**
npm scripts: mutation-test, mutation-test:util, build:test
A non-blocking CI workflow (weekly + workflow_dispatch) on node:20-bookworm that uploads the HTML report as an artifact
gitignores Stryker outputs

Baseline on `util/`

File	Mutation score (covered)	Notes
`util/index.ts`	57.69%	the only util file with tests (90 killed / 66 survived)
rest of `util/`	0%	no tests yet

This is exactly the signal mutation testing gives: it shows where a bug could be introduced and slip past the suite.

One integration note

Stryker builds in a clean sandbox, which surfaced that tsconfig.test.json (include tests/**/*) does not compile dynamically require()d files such as bot/modules/block/commands.ts — it only works normally because CI runs a full tsc first. Rather than touch that code, I use a build:test script (full tsc + test tsc) as Stryker's build command so the sandbox build is complete.

Validation

Normal test suite green (144 passing — no source touched)
tsc, lint, prettier clean
Mutation run completes locally in ~30s on util/

Summary by CodeRabbit

Chores
- Improved automated mutation testing in CI with updated repository checkout credentials handling.
- Continued running mutation testing weekly on a schedule and via manual execution.
- Added dedicated mutation-testing scripts, including a test build step.
- Enhanced mutation testing configuration by enabling TypeScript checking.
- Updated test artifact handling to upload mutation reports and ignore generated mutation/testing outputs locally.

Sets up mutation testing to measure how effectively the test suite catches bugs, scoped to util/ as the initial baseline (issue lnp2pBot#760). - Add @stryker-mutator core + mocha-runner + typescript-checker - stryker.config.json: mocha runner, perTest coverage, non-blocking (break: null), mutate util/** - npm scripts: mutation-test, mutation-test:util, build:test - Non-blocking CI workflow (weekly + manual) on node:20-bookworm that uploads the HTML report as an artifact - gitignore Stryker outputs The build:test script (full tsc + test tsc) is used as Stryker's build command so the sandbox build is complete: tsconfig.test.json alone does not compile dynamically required files (e.g. bot/modules/block/commands.ts), which only works in CI because it runs a full tsc beforehand. Baseline on util/: 57.69% mutation score on covered code in util/index.ts (the only tested util file); the rest of util/ has no tests yet. Part of lnp2pBot#760

coderabbitai · 2026-06-26T05:39:59Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0e8775d3-df3f-4ac0-b3fb-c2f8297c6423

📥 Commits

Reviewing files that changed from the base of the PR and between 6e3a996 and 568d95f.

📒 Files selected for processing (1)

.github/workflows/mutation-testing.yaml

Walkthrough

Adds Stryker mutation-testing setup for the util/ scope, including npm scripts, configuration, ignore rules, and a GitHub Actions workflow that runs weekly or manually and uploads the mutation report.

Changes

Mutation testing setup

Layer / File(s)	Summary
Configuration and scripts `package.json`, `stryker.config.json`, `.gitignore`	Adds mutation-testing scripts and Stryker dev dependencies, configures TypeScript mutation checking for `util/*/.ts`, and ignores generated outputs and command logs.
Mutation-testing workflow `.github/workflows/mutation-testing.yaml`	Defines a scheduled/manual workflow that runs the util mutation test in a Node 20 container and uploads `reports/mutation/` as `mutation-report`.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

Issue 760 — Covers the same StrykerJS mutation-testing setup, including scripts, configuration, util scope, and CI workflow.

Poem

I hopped through util paths by moonlit glow,
With Stryker nibbling where the mutants grow.
A cron bell rang; the report took flight,
A carrot-bright build in the midnight light. 🐰

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding a StrykerJS mutation-testing foundation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/mutation-testing.yaml:
- Around line 26-40: The workflow still uses mutable GitHub Action tags in the
mutation-testing job, so update the actions referenced by actions/checkout and
actions/upload-artifact to their full 40-character commit SHAs and keep the
original version as an inline comment for readability. Make this change in the
mutation-testing workflow alongside the existing run steps, preserving the same
behavior while removing tag-based supply-chain risk.

In `@stryker.config.json`:
- Around line 1-19: Stryker is not invoking the installed TypeScript checker
because the configuration lacks the explicit checker activation. Update the
Stryker config to enable the TypeScript checker via the checkers setting in the
existing JSON alongside the current testRunner, mutate, and thresholds options.
Keep the rest of the StrykerJS configuration unchanged so mutation testing still
runs with type checking enabled.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: a33067e2-79d9-478e-940f-d9d29f1cb9c0

📥 Commits

Reviewing files that changed from the base of the PR and between bf32684 and 542ad3f.

⛔ Files ignored due to path filters (1)

package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (4)

.github/workflows/mutation-testing.yaml
.gitignore
package.json
stryker.config.json

Matobi98 · 2026-07-29T22:20:28Z

Great work! Please address CodeRabbit's comments before merging.

- Pin GitHub Actions to full commit SHAs (actions/checkout, upload-artifact) with the version tag kept as an inline comment, removing tag-based supply-chain risk. - Enable the StrykerJS TypeScript checker (checkers + tsconfigFile) so type-invalid mutants are rejected before running tests. The @stryker-mutator/typescript-checker plugin was already a devDependency.

ToRyVand · 2026-07-30T02:45:43Z

Thanks @Matobi98! Both CodeRabbit points are addressed in 6e3a996:

SHA-pinned actions/checkout and actions/upload-artifact to full 40-char commit SHAs, with the version tag kept as an inline comment.
Enabled the TypeScript checker (checkers + tsconfigFile) — the @stryker-mutator/typescript-checker plugin was already a devDependency. Verified locally: Stryker runs clean with the checker active (util/index.ts: initial run passed, mutation completed end-to-end).

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/mutation-testing.yaml:
- Line 26: Update the actions/checkout step in the mutation-testing workflow to
set persist-credentials to false, ensuring checkout does not retain the GitHub
token while leaving the pinned action version and subsequent steps unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ddc4b2ab-c03b-4f11-afa5-636ec5c4862e

📥 Commits

Reviewing files that changed from the base of the PR and between 542ad3f and 6e3a996.

📒 Files selected for processing (2)

.github/workflows/mutation-testing.yaml
stryker.config.json

The mutation-testing job only reads the repo (no push), so the checkout should not retain the GitHub token in git config. Addresses CodeRabbit.

ToRyVand · 2026-07-30T03:00:55Z

Done in 568d95f — added persist-credentials: false to the checkout step. This job only reads the repo (no push), so there's no need to retain the token in git config. Thanks @coderabbitai.

coderabbitai · 2026-07-30T03:01:10Z

Tip

For best results, initiate chat on the files or code changes.

@ToRyVand Thanks for addressing this. Disabling persisted checkout credentials is appropriate for this read-only workflow. 🐇

If you found this review helpful, would you consider giving us a shout-out on X?

Thank you for using CodeRabbit!

_{You are interacting with an AI system.}

coderabbitai Bot reviewed Jun 26, 2026

View reviewed changes

Comment thread .github/workflows/mutation-testing.yaml Outdated

Comment thread stryker.config.json

coderabbitai Bot reviewed Jul 30, 2026

View reviewed changes

Comment thread .github/workflows/mutation-testing.yaml

chore: set persist-credentials false on checkout in mutation workflow

568d95f

The mutation-testing job only reads the repo (no push), so the checkout should not retain the GitHub token in git config. Addresses CodeRabbit.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: add StrykerJS mutation testing foundation - #847

test: add StrykerJS mutation testing foundation#847
ToRyVand wants to merge 3 commits into
lnp2pBot:mainfrom
ToRyVand:feat/issue-760-mutation-testing-stryker

ToRyVand commented Jun 26, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jun 26, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Matobi98 commented Jul 29, 2026

Uh oh!

ToRyVand commented Jul 30, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

ToRyVand commented Jul 30, 2026

Uh oh!

coderabbitai Bot commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ToRyVand commented Jun 26, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this adds

Baseline on util/

One integration note

Validation

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Matobi98 commented Jul 29, 2026

Uh oh!

ToRyVand commented Jul 30, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ToRyVand commented Jul 30, 2026

Uh oh!

coderabbitai Bot commented Jul 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ToRyVand commented Jun 26, 2026 •

edited by coderabbitai Bot

Loading

Baseline on `util/`

coderabbitai Bot commented Jun 26, 2026 •

edited

Loading