feat(2024): finalize generated monster data by bagelbits · Pull Request #1231 · 5e-bits/5e-database

bagelbits · 2026-09-22T21:37:48Z

What does this do?

Stacked on #1228, which is stacked on #1226. Retarget to main after #1226 and #1228 both merge.

This is the final step: land the generated data and remove everything that generated it.

Replace 5e-SRD-Monsters.json (previously 3 hand-authored monsters) with the full 341-monster generated output from feat(2024): validate and reshape generated monster data #1228, including the second audit pass's damage/condition immunities fix.
Delete the generator script (processMonsterData.mjs and its monster*.mjs modules), its raw inputs (monsters.json, monster-text-data.txt), its normalized-text output, and its own test (monsterGeneration.test.ts). None of these are needed once the data they produce is committed.
Remove tables.test.ts's bypass for the raw gist file, now that the file it skipped no longer exists.

How was it tested?

Full existing 2024 test suite (schema validation, duplicate indices, broken API references, translations) run against the final file with no bypass: 92 tests pass.

Is there a Github issue this is resolving?

NA

Did you update the docs in the API? Please link an associated PR if applicable.

No. A separate 5e-srd-api PR is needed for the new/changed 2024 monster shape (armor_class.type now optional, skills folded into proficiencies, new usage/spellcasting/damage/bonus_actions fields, and so on); not filed yet.

🤖 Generated with Claude Code

…2d94107f5533ba9c85c

… `-NYI`

Normalize the PDF text (ability scores, wrapped and merged header lines) and pair text blocks to monsters by ability scores plus CR instead of array position, which had swapped several monsters' skills and PB. Reshape output to MonsterSchema: numeric CR, xp/xp_in_lair, armor_class array, condition references, senses parsing, dc and multi-type damage parsed from descriptions. Add optional damage to reactions. Add monsterGeneration test that runs the script into a temp dir and checks schema, PB vs CR, skill bonuses, senses, XP, DC and damage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Read At Will and N/Day spell lists from the stat block text, since the gist descriptions stop at the intro line. Match each list to its entry by the intro's ability, save DC and to-hit bonus, because the PDF layout can put one block's spellcasting inside the previous block's text. Resolve every spell against 5e-SRD-Spells.json, map "(level N version)" to the spell level and other parentheticals to capitalized notes. Fail the script on any unknown spell or unused list. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Validate the generated monsters against the real schema instead of a test-only extension. Omit skills and gear when the source says None. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Build spellcasting for entries that cast named spells inline, including Coven Magic and bonus actions, reactions and legendary actions. Inherit ability, DC and to-hit from the monster's Spellcasting entry when the text says "the same spellcasting ability". Take per-day usage from the action name. Allow spellcasting on reactions and legendary actions in the schema. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Stacked PRs target a non-main base, so the main-only filter skipped lint and tests on them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Drop spell list lines the gist parsed as entries, strip the next block's title from the last entry of a block, split entries the gist merged, and complete descriptions truncated at page breaks from the stat block text. A completion is only accepted when the gist text is an exact prefix of the text paragraph. Link Nightmare Haunting's Dream cast. chain-devil Unnerving Gaze is cut off in the source text itself and stays listed as a known truncation in the test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The gist split the reaction in two. The text has the full sentence; the em dash in the gist stopped the prefix match. Normalize dashes and drop the known-truncation allow-list from the test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Move text normalization, stat block parsing, entry cleanup and spellcasting out of processMonsterData.mjs. Generated JSON and normalized text are byte-identical before and after. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Parse Multiattack descriptions into multiattack_type, actions and action_options, using the same Choice shape as the 2014 and existing 2024 monsters. Handle fixed sequences, ability uses, alternatives, "any combination" choices and "can replace one attack with" clauses. Attack type comes from the monster's own entry, or the spell for casts. Shapes that do not fit stay as plain descriptions and are listed in the test: any-combination plus replacement, and three one-off phrasings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

In a Choice, an action option's count is the most times it can be picked, so "three attacks, Claw or Tail in any combination" is choose 3 with each option at count 3. Replacements join the choice with count 1 (or the number of attacks they replace), which structures the remaining any-combo plus replacement entries, oni, dryad and guardian naga. Description text is unchanged. Also split Hand Crossbow (... Form Only) out of the wererat's Scratch, where the gist had merged them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Mirror the 2014 data: werebear, wereboar, wererat, weretiger and werewolf become human, hybrid and animal forms; vampire becomes vampire, bat and mist. Each form links the others through forms and reuses the 2014 index, name and image. Entries tagged "Form Only" go only to their forms and lose the tag. Multiattack options for actions a form lacks are dropped. Sizes come from Shape-Shift; speeds come from the Speed line in the text (the gist had the animal-form value) or from Shape-Shift for the vampire's bat and mist. Also merge fake entries the gist made from the tail of a Failure: or Success: line, such as the lycanthropes' curse and steam mephit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The 2024 SRD has no text after the AC value, so armor_class.type does not exist in the source. Make it optional in the schema and stop emitting the natural/armor guess. Link armor and shields named in a monster's Gear line to their 2024 equipment entries through armor_class.armor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…leed The gist splits "Medium or Small Humanoid" at the wrong point for 47 monsters, leaving size as just "Medium" and type as "or small humanoid". Repair both from the gist's own fields before the form split, so lycanthrope and vampire forms that need a single size still get one. 13 monsters have a languages string truncated mid-sentence in the gist (dretch, kraken, and others); read it from the stat block text instead, where the header-wrapping normalizer already joins it into one line. The tarrasque's World-Shaking Movement description was already cut short in the gist ("...next turn. Toughs", a page-break artifact), which failed the completeness check and triggered completeFromText's repair pass. That pass had no way to recognize "Toughs" as a title, since it has no AC line of its own, so it kept accumulating text straight through the next stat block's ability-score table, gear, senses, languages and CR line. Stop the repair at the ability-score table header, which every stat block has, regardless of whether its title was recognized, and recognize a title followed directly by that header (no AC/HP/Speed of its own) as a title too. Found by an independent audit subagent; see the four findings it confirmed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The gist leaves an entry's usage as unparsed text in its name, e.g. "Legendary Resistance (3/Day, or 4/Day in Lair)" or "Acid Breath (Recharge 5-6)", and the schema's usage field (present on the 3 hand-authored reference monsters) was left empty for all 341 monsters. Parse per day, per day in lair (special abilities only), recharge on roll, and recharge after a rest, matching the existing 2014/reference shapes exactly. A trailing note that isn't itself a usage (night hag's "Requires Soul Bag") is kept in the name; a suffix that isn't a usage at all (a spell's "(level N version)", "(Requires Magic Rope)", the mimic's leftover form tag) is left untouched. Add rest_types to ActionUsageSchema and usage to Reaction/LegendaryAction, since the data has these on reactions and bonus actions, not just special abilities and actions. Found by an independent audit subagent (finding #2). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2014 monsters have no separate skills field: a skill is just another proficiencies entry, alongside saving throws. Convert the "History +12, Perception +10" string the same way instead of keeping it as a field no 2014 or reference 2024 monster carries. Drop skills from the schema. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

440 attack entries had no attack_bonus even though the schema supports it and the reference aboleth's Tentacle has one. The gist's own attack_bonus was a leftover spell-attack modifier on 5 dragons' Spellcasting action (already covered correctly by spellcasting.modifier), never a weapon attack roll bonus, so it is dropped in favor of parsing "Attack Roll: +N" from the description, same as damage and dc. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The gist's own "Immunities" extraction lands a conditions-only line (no damage type present, so no semicolon) whole in damage_immunities, leaving condition_immunities empty. Confirmed for 23 monsters (gray-ooze, ettin, cloaker, and 20 others). Read Resistances/Vulnerabilities/ Immunities from the stat block text instead, classifying an unsemicoloned Immunities line word-by-word against the known damage types and conditions. This also repairs 4 monsters (ghost, specter, will-o-wisp, wraith) whose damage_resistances was truncated with a trailing empty string, and rakshasa/half-dragon's single-phrase vulnerability/resistance. Also capture a condition's parenthetical note (e.g. archmage's "Charmed (with Mind Blank)") into APIReferenceSchema's existing optional `note` field instead of discarding it. Found by a second independent audit subagent, scoped to the final PR. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…data-validation-ad996c # Conflicts: # src/2024/en/5e-SRD-Monsters-New.json # src/2024/en/processMonsterData.mjs

Replace the 3-monster 5e-SRD-Monsters.json with the full 341-monster generated output, verified against every existing 2024 test (schema, duplicate indices, broken references, translations). Remove the generator script and its modules, the raw gist and PDF-text inputs, the normalized text output, and the generator's own test, none of which are needed once the data they produce is committed. Drop tables.test.ts's temporary bypass for the raw gist file, which no longer exists. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Pull in the fix from the parent PR: 23 monsters had a conditions-only "Immunities" line landing whole in damage_immunities instead of condition_immunities, 4 had a truncated damage_resistances, and condition parenthetical notes (e.g. archmage's "with Mind Blank") were being discarded instead of kept as APIReferenceSchema's note field. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

## What does this do? Stacked on #1226. Base branch `add2024Monsters-base` is copy of that PR head. Retarget to `main` after #1226 merge. Script output now match `MonsterSchema`. New test check it. CI now run on PRs to any base branch, so stacked PRs get lint + tests. - Normalize PDF text: ability scores on one line, wrapped header lines joined, merged lines split. Write `monster-text-data.normalized.txt`. - Pair text block to monster by ability scores + CR, not array position. Old way swap skills/PB for 7 monsters (giant-crab, giant-crocodile, winter-wolf, worg, allosaurus, ankylosaurus, ape). - Reshape: numeric CR, `xp`, `xp_in_lair`, `armor_class` array, condition references, senses parsing, hit dice minus sign. - Parse `damage` (all types, one entry each) and `dc` (saving throw, escape, ability check) from descriptions for traits, actions, bonus actions, reactions, legendary actions. - Build `spellcasting`. List entries (48): spell lists come from stat block text, gist stop at intro line; match list to entry by ability + save DC + to-hit since PDF put some blocks inside previous block text. Single-spell casts (Invisibility, Fear, Coven Magic, Divine Aid, and similar): link spell, inherit ability/DC from main Spellcasting when text say "same spellcasting ability", per-day usage from action name. Every spell resolve against `5e-SRD-Spells.json`; script throw on unknown spell or unused list. `(level N version)` set spell level, other parentheticals become capitalized `notes`. - Clean PDF debris from entries: drop spell list lines gist parsed as entries, strip next block title from last entry, split merged entries, complete descriptions cut at page breaks from stat block text (only when gist text is exact prefix of text paragraph). 1460 entries down to 1330. - Schema: add optional `damage` to `ReactionSchema`. Add optional `spellcasting` to reactions and legendary actions. Add optional `skills`, `gear`, `bonus_actions` to `MonsterSchema`. Omit `skills`/`gear` when source say None. - Split script into modules: `monsterText.mjs` (normalize, parse blocks, pair to monsters), `monsterEntries.mjs` (damage, dc, debris cleanup), `monsterSpellcasting.mjs`, `monsterCommon.mjs`. Output byte-identical before and after split. - Structure all 178 `Multiattack` entries: `multiattack_type`, `actions`, `action_options`. Same Choice shape as 2014 and existing 2024 monsters. Attack `type` from the monster's own action (melee/ranged/special), or the spell's `attack_type` for casts. Description text unchanged. - New convention for "any combination": inside a Choice, an action option `count` is the most times that option can be picked. "Three attacks, Claw or Tail in any combination" = `choose: 3`, Claw `count: 3`, Tail `count: 3`. Replacements join the choice (count = attacks replaced). No existing data use `count` > 1 inside `choose` > 1, so this is new. Please check it fits API consumers. - Split form monsters like 2014: werebear, wereboar, wererat, weretiger, werewolf into human/hybrid/animal, vampire into vampire/bat/mist. 329 monsters become 341. Same index, name and image as 2014, linked by `forms`. Entries tagged "Form Only" go only to their forms (tag removed), untagged go to every form. Multiattack options pruned to actions the form has. Size from Shape-Shift. Speed from text `Speed` line (gist had animal-form value, e.g. werewolf 40 ft. for every form) or from Shape-Shift for vampire bat and mist. Human and vampire form keep base size "Medium or Small". - Merge fake entries gist made from tail of `Failure:`/`Success:` lines (lycanthrope curse, steam mephit Steam Breath). - `armor_class`: no `type` (2024 SRD has no text after AC value, per review comment). `type` now optional in schema. Armor and shields named in `Gear` link to 2024 equipment through `armor_class.armor` (28 monsters). Half Plate Armor matches Half-Plate Armor. - Independent audit (subagent) found and confirmed 4 issues, now fixed: - `size`/`type` split wrong for 47 monsters ("Medium or Small Humanoid" -> size "Medium", type "or small humanoid"). Split repaired from the gist's own two fields; form-split monsters (finding #1) still get a single size per form. - `languages` truncated mid-sentence for 13 monsters in the gist (dretch, kraken, and 11 others); now read from the stat block text instead. - Tarrasque's "World-Shaking Movement" legendary action had ~150 characters of the next stat block's ability scores, gear, senses and CR line bled in, because that block's title ("Toughs") has no AC line of its own so went undetected. Fixed generally: the truncation-repair pass now always stops at the ability-score table header, and a title followed straight by that header (no AC/HP/Speed) is now recognized too. - `usage` (finding #2, 0 -> 150 entries): parsed out of recharge/per-day name suffixes. "Legendary Resistance (3/Day, or 4/Day in Lair)" -> name "Legendary Resistance", `usage: {type: "per day", times: 3, times_in_lair: 4}`, matching the 3 reference monsters exactly. Covers per day, per day in lair (special abilities only), recharge on roll (dice always "1d6"), recharge after a rest. Non-usage suffixes ("level N version", "Requires Magic Rope", mimic's form tag) left alone; a trailing non-usage note (night hag "Requires Soul Bag") kept in the name. Added `rest_types` to `ActionUsageSchema` and `usage` to `Reaction`/`LegendaryAction` schemas, since the data has recharge/per-day on reactions and bonus actions too, not just special abilities and actions. - `skills`/`proficiencies` (finding #5): decided in favor of matching 2014. Skills now encoded as `proficiencies` entries (`skill-history`, `skill-perception`, and so on, looked up from `5e-SRD-Proficiencies.json`), alongside saving throws, same as every 2014 monster. `skills` string field dropped from output and schema. - `attack_bonus`: 440 attacks had none set, though schema support it and reference aboleth's Tentacle has one. Gist's own `attack_bonus` was a leftover spell-attack modifier on 5 dragons' Spellcasting action (already correct in `spellcasting.modifier`), never a weapon attack bonus; dropped, and parsed properly from "Attack Roll: +N" in the description, same as damage and dc. Found while comparing generated output against the reference monsters ahead of a final cleanup PR. - Second independent audit pass (subagent), scoped to the finalized PR (#1231): found and fixed 1 new critical issue plus 3 related truncations. - **`damage_immunities`/`condition_immunities`**: the gist's own "Immunities" extraction lands a conditions-only line (no damage type, so no semicolon in the source) whole in `damage_immunities`, leaving `condition_immunities` empty. Confirmed for 23 monsters (gray-ooze, ettin, cloaker, and 20 others). Fixed by reading `Resistances`/`Vulnerabilities`/`Immunities` from the stat block text, classifying an unsemicoloned Immunities line word-by-word against the known damage types and conditions. - `damage_resistances` truncated (trailing empty string) for 4 undead (ghost, specter, will-o-wisp, wraith); rakshasa/half-dragon's single-phrase vulnerability/resistance also truncated. Same fix. - Condition parenthetical notes (archmage's "Charmed (with Mind Blank)") were being discarded; now captured in `APIReferenceSchema`'s existing optional `note` field. - Regenerate `5e-SRD-Monsters-New.json`. ## How was it tested? New `src/2024/tests/monsterGeneration.test.ts`. Run script into temp dir, no overwrite of real files. Check: count + unique index, schema, hit dice format, passive perception, sense format, XP, PB vs CR, skill bonus = mod + PB or 2 x PB, every save DC and damage roll in description present in output, every spell cast has valid spell URL, no spell list entries or unterminated descriptions. Every Multiattack must be structured and simple counts must round trip. Forms split, linked and pruned; werewolf human/wolf entries and speeds checked. Armor links checked against equipment. Full suite 113 pass (7 new tests total for both audit passes). ## Known gaps - `spell save DC` not put in `dc`. It live in `spellcasting.dc`. - Dryad Multiattack read as one choice of Vine Lash, Thorn Burst or Charm Monster. Text say "and it can use Spellcasting" so could mean attack plus cast. - Clay golem Multiattack condition ("if it used Hasten this turn") only in Choice `desc`, not structured. - `components_required` computed from text: "no Material components" give `["V","S"]`. - Reference adult-blue-dragon (`5e-SRD-Monsters.json`) has extra `acid-arrow` and `modifier: 5`. Book text has neither. Output follow book text. - Charmed (with Mind Blank) on archmage become plain `charmed`. - Escape DC and first-named ability of check DC map to one ability. Book allow choice. - shambling-mound and giant-frog Stealth match book, not mod + PB. Listed as quirks in test. - Mist form gets every untagged entry (Charm, legendary actions, traits) since tags followed strictly. Book say mist form can't take actions. - Mimic `Adhesive (Object Form Only)` keeps tag. Not split in 2014. - Some words keep gist compound artifacts like `5-footwide`. - Merged-entry split also cover `(... Form Only)` headings (wererat Hand Crossbow). ## Is there a Github issue this is resolving? NA ## Did you update the docs in the API? Please link an associated PR if applicable. No 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: G.Ambatte <sean@robertson-family.nz> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

…final-monster-data # Conflicts: # src/2024/en/5e-SRD-Monsters-New.json # src/2024/en/processMonsterData.mjs

🤖 I have created a release *beep* *boop* --- ## [5.12.0](v5.11.1...v5.12.0) (2026-09-22) ### Features * **2024:** Add 2024 monsters ([#1229](#1229)) ([cb70dd4](cb70dd4)) * **2024:** finalize generated monster data ([#1231](#1231)) ([6f6299e](6f6299e)) * **2024:** validate and reshape generated monster data ([#1228](#1228)) ([8693259](8693259)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: 5e-bits-deploy-bot[bot] <209387221+5e-bits-deploy-bot[bot]@users.noreply.github.com>

bagelbits changed the base branch from pr-data-validation-base to claude/pr-data-validation-ad996c September 22, 2026 21:47

bagelbits added this pull request to stack #1232 September 22, 2026 21:47

bagelbits force-pushed the claude/pr-final-monster-data branch from 20d2612 to d12b8e2 Compare September 22, 2026 22:12

bagelbits mentioned this pull request Sep 22, 2026

feat(2024): validate and reshape generated monster data #1228

Merged

G-Ambatte and others added 26 commits September 22, 2026 15:19

Add monster data from https://gist.github.com/krisimmig/f4b7d2a30cadd…

4ecd85f

…2d94107f5533ba9c85c

Add function to process monster data and resulting file

53c4f2f

Use existing image files where present, mark non-existing images with…

c619333

… `-NYI`

Initial attempt at calculating proficiency_bonus

1a37c2e

Fix broken data in source file

991f2d7

Updated output file

bc4245e

Fix tests

cae5169

Fix source data senses (Darkvision unimpeded by magical darkness)

803fb29

Add text extraction of SRD Monster data

4542e25

Add text extraction to processing function

2d6ad6a

Update resulting file

0615fa4

feat(2024): add skills, gear and bonus_actions to monster schema

ccc3c64

Validate the generated monsters against the real schema instead of a test-only extension. Omit skills and gear when the source says None. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

ci: run CI on pull requests to any base branch

8dd8b34

Stacked PRs target a non-main base, so the main-only filter skipped lint and tests on them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

docs(schema): document count on action options in a Choice

658757c

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

bagelbits and others added 3 commits September 22, 2026 15:19

bagelbits force-pushed the claude/pr-final-monster-data branch from d12b8e2 to 809e735 Compare September 22, 2026 22:19

bagelbits and others added 3 commits September 22, 2026 15:46

Merge branch 'main' of github.com:5e-bits/5e-database into claude/pr-…

358e249

…data-validation-ad996c # Conflicts: # src/2024/en/5e-SRD-Monsters-New.json # src/2024/en/processMonsterData.mjs

bagelbits force-pushed the claude/pr-final-monster-data branch from 809e735 to b77a2c1 Compare September 22, 2026 22:47

Base automatically changed from claude/pr-data-validation-ad996c to main September 22, 2026 22:47

Merge branch 'main' of github.com:5e-bits/5e-database into claude/pr-…

3c2ae1e

…final-monster-data # Conflicts: # src/2024/en/5e-SRD-Monsters-New.json # src/2024/en/processMonsterData.mjs

bagelbits merged commit 6f6299e into main Sep 22, 2026
6 checks passed

bagelbits deleted the claude/pr-final-monster-data branch September 22, 2026 22:53

5e-bits-deploy-bot Bot mentioned this pull request Sep 22, 2026

chore(main): release 5.12.0 #1233

Merged

Navigation Menu

Uh oh!

feat(2024): finalize generated monster data - #1231

feat(2024): finalize generated monster data#1231
bagelbits merged 33 commits into
mainfrom
claude/pr-final-monster-data

bagelbits commented Sep 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bagelbits commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this do?

How was it tested?

Is there a Github issue this is resolving?

Did you update the docs in the API? Please link an associated PR if applicable.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bagelbits commented Sep 22, 2026 •

edited

Loading