fix: treat Indic conjunct clusters as a single grapheme (v7) - #3151
Merged
Conversation
`getCharacterDistance` implemented the Unicode UAX 29 grapheme rules through GB11 and GB12/13 but predated GB9c, so an Indic conjunct cluster (consonant + virama + consonant, e.g. Devanagari क्ष) was split at the virama: caret movement and character positions treated the cluster as two characters where browsers render one glyph. Port the GB9c implementation from upstream Slate (ianstormtaylor/slate#6074): `InCB` Consonant/Extend/Linker codepoint classes, a GB9c entry in `NonBoundaryPairs`, and a deferred `endsWithConjunctLinker` check that only suppresses the boundary when the left context actually contains a conjunct linker. The character tables are identical to upstream's; they are composed into `RegExp`s from shared class strings instead of upstream's inline literals, so each table ships once (~6 KB of bundled Unicode data instead of ~18). Every consumer of `getCharacterDistance` (caret movement, positions, deletion units) now steps over the full cluster. Forward delete at the start of a cluster now removes the whole cluster. Backspace stays at one code unit per press: the whole-cluster delete composes with the compound-script reinsert in `delete-internal.ts`. That reinsert was keyed to script ranges that miss Gujarati, Tai Tham, Balinese, Sundanese, Javanese, and Meetei Mayek, which GB9c now glues, so `COMPOUND_SCRIPT_REGEX` additionally matches those scripts' BMP conjunct linkers (every GB9c cluster contains its linker). The supplementary-plane linkers are left out deliberately: the reinsert slices code units and would split a surrogate pair, so backspace on those scripts removes the whole cluster. Pinned by unit tests adapted from upstream (LTR and RTL, red on the pre-GB9c code) and browser tests: forward delete over a cluster (red pre-GB9c) and backspace one-code-unit parity for Devanagari (in-range script) and Gujarati (linker-keyed, red without the regex extension). (cherry picked from commit 78360a0)
Backspace on an Indic conjunct cluster removes one code unit at a time (the compound-script reinsert in `delete-internal.ts`), but that override is keyed to a hardcoded set of script ranges. Pin that a ZWJ emoji sequence, which matches none of them, is removed whole by a single backspace. (cherry picked from commit dac0376)
🦋 Changeset detectedLatest commit: f51b78a The changes in this PR will be included in the next version bump. This PR includes changesets to release 14 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📦 Bundle Stats —
|
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 821.9 KB | +7.5 KB, +0.9% |
| Internal (gzip) | 184.0 KB | +2.9 KB, +1.6% |
| Bundled (raw) | 1.43 MB | +7.5 KB, +0.5% |
| Bundled (gzip) | 352.0 KB | +3.1 KB, +0.9% |
| Import time | 62ms | -5ms, -7.1% |
@portabletext/editor/behaviors
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 4.2 KB | - |
| Internal (gzip) | 1.4 KB | - |
| Bundled (raw) | 3.9 KB | - |
| Bundled (gzip) | 1.3 KB | - |
| Import time | 1ms | -0ms, -11.5% |
@portabletext/editor/plugins
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 5.2 KB | - |
| Internal (gzip) | 1.8 KB | - |
| Bundled (raw) | 5.0 KB | - |
| Bundled (gzip) | 1.7 KB | - |
| Import time | 5ms | -0ms, -4.8% |
@portabletext/editor/selectors
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 96.4 KB | - |
| Internal (gzip) | 22.3 KB | - |
| Bundled (raw) | 92.5 KB | - |
| Bundled (gzip) | 21.0 KB | - |
| Import time | 5ms | -0ms, -6.1% |
@portabletext/editor/traversal
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 40.7 KB | - |
| Internal (gzip) | 10.6 KB | - |
| Bundled (raw) | 41.1 KB | - |
| Bundled (gzip) | 10.6 KB | - |
| Import time | 4ms | -0ms, -8.3% |
@portabletext/editor/utils
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 34.1 KB | - |
| Internal (gzip) | 8.7 KB | - |
| Bundled (raw) | 32.5 KB | - |
| Bundled (gzip) | 8.5 KB | - |
| Import time | 4ms | -0ms, -6.0% |
🗺️ . · ./behaviors · ./plugins · ./selectors · ./traversal · ./utils · Artifacts
Details
- Import time regressions over 10% are flagged with
⚠️ - Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.
📦 Bundle Stats — @portabletext/markdown
Compared against editor-v7.x (49ada3d6)
| Metric | Value | vs editor-v7.x (49ada3d) |
|---|---|---|
| Internal (raw) | 53.0 KB | - |
| Internal (gzip) | 11.6 KB | - |
| Bundled (raw) | 348.0 KB | - |
| Bundled (gzip) | 98.2 KB | - |
| Import time | 26ms | +2ms, +7.8% |
🗺️ View treemap · Artifacts
Details
- Import time regressions over 10% are flagged with
⚠️ - Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.
christianhg
marked this pull request as ready for review
August 24, 2026 11:41
Merged
christianhg
added a commit
that referenced
this pull request
Aug 25, 2026
Documents how a fix on `main` ships for the v6/v7 lines: the `editor-v<N>.x` branch layout with their self-triggering release pipelines, the verify-adapt-red-verify procedure, version-suffixed changesets, and the backport commit/PR shape, distilled from the real backports (#2919, #2872, #3151, #3120). Same layout as the existing skills: the file lives in `.agents/skills/`, with a committed `.claude/skills` symlink for Claude Code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #3136 to the v7 line; both commits cherry-pick clean (the touched files are identical on the two branches apart from a lint directive). See #3136 for the full narrative: the grapheme walker gains UAX #29 rule GB9c so Indic conjunct clusters move as one character, forward delete removes a whole cluster, backspace keeps its one-code-unit steps via the compound-script reinsert, and the Unicode tables ship once as composed
RegExps.All gates and both delete browser suites pass on this branch (unit 1381/1381, browser 18/18).
Note
Medium Risk
Changes grapheme-boundary and character-delete behavior for Indic scripts, so caret and Delete/Backspace can mis-step if GB9c or the compound-script reinsert is wrong.
Overview
Makes Indic conjunct clusters (consonant + virama + consonant, e.g. Devanagari क्ष) a single grapheme in
getCharacterDistance, following Unicode UAX #29 GB9c. Caret movement no longer lands inside the cluster; Delete at the start removes the whole cluster.Backspace still removes one code unit at a time via the existing compound-script reinsert.
COMPOUND_SCRIPT_REGEXnow also matches BMP conjunct linkers that the old script ranges missed (e.g. Gujarati). Supplementary-plane linker scripts are left out so reinsert does not split surrogates; those clusters delete as a whole on backspace.Unicode InCB tables are shared across the new regexes so they ship once. Tests cover GB9c clustering, Indic/Gujarati backspace, ZWJ emoji (still whole-cluster), and forward delete of a conjunct.
Reviewed by Cursor Bugbot for commit f51b78a. Bugbot is set up for automated code reviews on this repo. Configure here.