diff --git a/CHANGELOG.md b/CHANGELOG.md index e970853..3891326 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,7 @@ Leafdown uses lightweight [Keep a Changelog](https://keepachangelog.com/en/1.1.0 ### Fixed +- Read a typed `*` or `_` run the way Markdown reads the same characters in a file, so `***text*` gives two literal asterisks before italic text, `_**text**` a literal underscore before bold text, and `_**text**_` italic bold, instead of leaving every marker as text that saved with backslashes and reopened without the formatting. A run whose closing marker is shorter than its opening one is read once the caret leaves it, because another marker typed there would spell something else. - Pair a `*`, `_`, or `~` typed against bold, italic, or strikethrough with the matching literal marker already on the other side of it, so closing `_**text**` with a `_` gives italic bold and saves `_**text**_`, instead of leaving both markers as text that saved as `\_**text**\_` and reopened without the italic. A marker a file keeps literal by escaping it stays literal. - Leave a `*`, `_`, or `~` bare on save wherever nothing else on its line could pair with it, so text such as `[a](b)*` keeps its marker as written instead of collecting a backslash merely because a link, an image, or a bold span shares the line with it. A marker that could still pair, including one that could pair with the markers of a span beside it, keeps its backslash. - Keep the address of a URL or email address written on its own when a `*`, `_`, or `~` follows it, so text such as `https://example.com*` keeps its link pointing where it did. The backslash the file writes to keep that marker literal was being read back as part of the address, which gained another backslash every time the document was opened and saved. diff --git a/docs/decisions.md b/docs/decisions.md index 0991b5c..d5a639f 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -145,6 +145,8 @@ - The preset's strikethrough delimiter run is not preserved. Its strikethrough mark carries no marker attribute, unlike emphasis and strong, so a single-tilde run parses and serializes back as a double-tilde run. This is left as a known normalization under [Preserve Markdown semantics over exact formatting](#preserve-markdown-semantics-over-exact-formatting) rather than overridden as the autolink form was, because both runs mean the same thing to a GFM reader. Preserving the authored run would require carrying the marker on the mark. - The preset's strikethrough input rule is overridden. Its `(~{1,2})` backtracks to a one-tilde delimiter run when no two-tilde closing run exists yet, and its content group does not exclude the marker, so typing `~~text~~` created a mark over `~text` on the seventh keystroke and left a surplus tilde on each side that saved as an escaped character. Leafdown carries its own rule, decided in [issue #233](https://github.com/Azganoth/leafdown/issues/233), which excludes the marker from the content and anchors the match at the caret so a run stays literal text until the author closes it. This is the only input rule Leafdown owns; every other preset rule either anchors at the caret or excludes its own marker, and none of them can match a run this way. - The replacement rule keeps the preset's leading word, colon, and slash guard, so a tilde run that touches one of those does not become a strikethrough as it is typed. `lead~~text~~` and `1~2~3` parse as strikethrough when a file holds them but stay literal text when typed, which is a real disagreement, examined in [issue #282](https://github.com/Azganoth/leafdown/issues/282) and left as it is. The guard does two jobs: it holds the word boundary, and it stops a one-tilde run from opening inside an unclosed two-tilde one. Removing it fixes the first case and breaks `~~a~b~~`, which types as a struck `b` between literal tildes, because an input rule reads only the text before the caret and cannot know another tilde is coming. Separating the two jobs means matching delimiter runs directly rather than through `markRule`, which reads one content group and cannot express the alternation. Leafdown prefers the conservative failure: literal text the author can see and correct, over a construct silently built around the wrong delimiters. +- A `*` or `_` run the preset input rules decline is paired by Leafdown instead, decided in [issue #232](https://github.com/Azganoth/leafdown/issues/232). Those rules read one content group between delimiters of equal length, so a run closed against an unequal or differently spelled one matched nothing and stayed literal: `***text*`, `_**text*`, `_**text**`, and `*__text_` kept every marker as text, and `_**text**_` closed an italic span over the literal asterisks its bold span should have spelled. The pairing that already answers a delimiter typed beside a mark now answers one typed beside literal text as well, over as many delimiters as the shorter run spells. It acts only where the preset declined, because a rule that matches consumes the typed character before the plugin sees it, which is why the preset's guards are reproduced rather than replaced and `lead**text**` still types literal. The underscore is the one exception: `\w` counts it as a word character where CommonMark counts it as punctuation, and `_**text**` cannot be read otherwise. +- The pairing waits where the closing run is shorter than the run it would pair with. An input rule reads only the text before the caret and cannot know another delimiter is coming, and here the continuations are each other: firing on the keystroke serves `***text*` at the cost of `***text**`, and refusing serves the second at the cost of the first. Reading the pair once the caret leaves the run it closed serves both, and is how typed link source already reaches the object it spells. A closing run that could equally open a construct is refused outright rather than deferred, on the conservatism above: `**a*b` leaves its delimiters as text the author can see and correct, rather than pairing a run they may still be typing into. A tilde is left out of this pass entirely, because a strikethrough is spelled only by runs of equal length. - Develop custom UI components only when required by the product specification. ### Do not use Crepe diff --git a/docs/specification.md b/docs/specification.md index 6ae2a0f..43fde39 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -153,6 +153,7 @@ The editor is a unified hybrid Markdown surface. Behavior is governed by renderi - Link and autolink projection exposes their source directly in the document; links preserve their label, target, optional title, and compatible uniform outer inline formatting. An autolink keeps the form it was authored in, bare or angle-bracket, when it is projected and when it is saved. A link remains one semantic projection owner. A caret or contained text selection anywhere in a supported label projects the complete link source, including labels with nested strong, emphasis, strikethrough, inline-code formatting, semantic soft line endings, an image, a footnote reference, or a preserved character reference. An image or footnote reference in a projected label becomes its own Markdown source and returns as its object when the label commits. Soft line endings remain one logical label; indentation follows Leafdown's canonical serialization. Valid edits rehydrate one link over the complete rich label; invalid or incomplete edits become the literal text the source spells, where a backslash escapes the character it precedes and is otherwise kept as text. Mixed-format and multiline labels do not fall back to fragmented projections for their nested content. - Text written in the current editing session that is exactly the source of one inline link, image, autolink literal, or URI autolink becomes that object when the caret leaves it, including when a line break ends the line it sits on, so typing and plain-text paste of the same characters reach the same document. The caret has not left while only source characters separate them, which keeps a bare URL whole as it is typed. Backslash-escaped source, incomplete source, and text that is not exactly one object's source stay literal. Source the file escaped is literal text the author asked for, so editing it leaves it literal and only the escape gesture below converts it, and `Undo` returns a committed link to the source it was written as. Because a committed link projects its source again whenever the caret returns, the visible text does not change. - A `*`, `_`, or `~` typed against a marked run pairs with a matching literal run of the same length on the other side of that run, so the two become the emphasis, strong emphasis, or strikethrough the line spells and neither delimiter is left as content. The pair forms only where the outer neighbours let both runs flank their content, which leaves a run sitting after a letter or a digit literal, and only where the span does not already carry the mark the pair would apply. Pairing answers the character typed in this session rather than the shape the document holds, so a file that keeps a delimiter on each side of a run literal by escaping it opens that way and stays that way. +- A `*` or `_` typed to close a run of literal text pairs with the run that same text opens, over as many delimiters as the shorter of the two spells and leaving the surplus literal, so `***text*` reads as two literal asterisks before italic text and `_**text**` as a literal underscore before bold text. Where the closing run is the shorter one the author may still be extending it, so the pair is read once the caret leaves that run rather than as the character lands. A run stays literal where a letter, a digit, a colon, or a slash sits against the run it would open, where the run it closes could equally open a construct the author has not finished, and wherever CommonMark's flanking rules refuse it. A tilde is not read this way, because a strikethrough is spelled only by runs of equal length. - Literal text the file keeps literal by escaping projects that escape when a caret or contained text selection reaches it, so the run reads as the file will write it. The gesture covers a run that spells exactly one inline link or image, which is what an escape there can be spent on; text kept literal in any other form shows no escape, because deleting one would change nothing. Deleting a projected backslash converts the run to the object it describes at once, without waiting for the caret to leave; that object then projects its own source in the escape's place, so the characters stay put and the caret keeps its position while the markers restyle around it. `Undo` reverses the conversion like any other document change, and typing the backslash back turns the object literal again. Leaving the escape alone restores the run unchanged, and an edit that neither keeps the escape nor spells one object becomes the literal text the source spells. Nothing durable records the escape; it is derived from the text and the serializer on every open, so the same gesture is available after a save and reopen. Source this session wrote keeps the commit rule above instead, so a run being typed never shows a backslash the author did not write. - A character reference the file preserves projects its authored source when a caret or contained text selection reaches it, so `©`, `©`, and `©` read as the file will write them rather than as the character they render. References written next to each other are each preserved and each projected on their own, whether they repeat, differ, or name more than one character, so a file holding `©©` keeps both and breaking one leaves its neighbour intact, saving `©©`. Entry direction follows the rule for atomic inline projections: a caret entering from the left starts at the beginning of the source, a caret entering from the right starts at the end, a position between two adjacent references enters the one that follows it, and a selection over the rendered character selects the complete source. Leaving the source unchanged restores the rendered character exactly. An edit that still spells one reference commits that reference, so a form can be rewritten in place; any other edit becomes the literal text the source spells, on the same escape rule as a link, which is how removing a character from the source removes the reference from the saved file. Nothing durable records the projection, and a stored source that no longer spells the text it covers is written as that text and projects nothing. - A character reference within one exact, contiguous supported mark combination belongs to that marked fragment. Entering through its text, the reference itself, or either end of the fragment projects one outer wrapper such as `**a©b**`, and a valid edit commits one mark with the reference intact. Logical links retain higher semantic ownership, so a caret or contained text selection anywhere in a label holding a reference projects the complete link source, while standalone or otherwise ineligible references use the reference-only projection above. diff --git a/src/features/editor/plugins/attentionPairing.test.ts b/src/features/editor/plugins/attentionPairing.test.ts index d63db66..7772b64 100644 --- a/src/features/editor/plugins/attentionPairing.test.ts +++ b/src/features/editor/plugins/attentionPairing.test.ts @@ -146,3 +146,73 @@ describe("pairing a delimiter typed beside a mark", () => { expect(settle(mounted)).toBe("text[emphasis] tail"); }); }); + +describe("typing a delimiter run into literal text", () => { + // The run needs a paragraph the tail does not share: a run closed against a letter reads as the + // opener of a construct the author has not finished, and stays literal. + const typeIntoEmptyParagraph = async (input: string) => { + const mounted = await mountEditor(`x${TAIL}`); + + mounted.view.dispatch(mounted.view.state.tr.delete(1, 2)); + setTextSelection(mounted.view, 1); + typeText(mounted.view, input); + + return mounted; + }; + + it.each([ + { document: "** text[emphasis]", saved: "\\*\\**text*", typed: "***text*" }, + { document: "_* text[emphasis]", saved: "_\\**text*", typed: "_**text*" }, + { document: "_ text[strong]", saved: "_**text**", typed: "_**text**" }, + { document: "*_ text[emphasis]", saved: "*\\__text_", typed: "*__text_" }, + { document: "text[emphasis+strong]", saved: "_**text**_", typed: "_**text**_" }, + { document: "* text[emphasis]", saved: "\\**text*", typed: "**text*" }, + { document: "*** text[emphasis]", saved: "\\*\\*\\**text*", typed: "****text*" }, + { document: "* text[strong]", saved: "\\***text**", typed: "***text**" }, + { document: "* text[strong]", saved: "*__text__", typed: "*__text__" }, + { document: "text[emphasis+strong]", saved: "***text***", typed: "***text***" }, + ])( + "reads $typed the way CommonMark reads the same source", + async ({ document, saved, typed }) => { + const mounted = await typeIntoEmptyParagraph(typed); + + expect(settle(mounted)).toBe(`${document} tail`); + expect(mounted.getMarkdown()).toBe(`${saved}\n\ntail\n`); + // The file spells the delimiters the author typed and no others, whether it writes them as + // syntax or escapes them as text. + expect(saved.replaceAll(/\\(?=[*_~])/gu, "")).toBe(typed); + + const reopened = await mountEditor(mounted.getMarkdown()); + + expect(describeDocument(reopened)).toBe(`${document} tail`); + }, + ); + + it("holds a run the author can still extend until the caret leaves it", async () => { + const mounted = await typeIntoEmptyParagraph("***text*"); + + expect(describeDocument(mounted)).toBe("***text* tail"); + + typeText(mounted.view, "*"); + + expect(settle(mounted)).toBe("* text[strong] tail"); + }); + + it("leaves a run closed against a letter literal", async () => { + const mounted = await typeIntoEmptyParagraph("**a*b"); + + expect(settle(mounted)).toBe("**a*b tail"); + }); + + it("leaves a run that follows a word literal", async () => { + const mounted = await typeIntoEmptyParagraph("lead**text**"); + + expect(settle(mounted)).toBe("lead**text** tail"); + }); + + it("leaves an unequal tilde run literal", async () => { + const mounted = await typeIntoEmptyParagraph("~~text~"); + + expect(settle(mounted)).toBe("~~text~ tail"); + }); +}); diff --git a/src/features/editor/plugins/attentionPairing.ts b/src/features/editor/plugins/attentionPairing.ts index c8de3ca..5c20979 100644 --- a/src/features/editor/plugins/attentionPairing.ts +++ b/src/features/editor/plugins/attentionPairing.ts @@ -5,11 +5,24 @@ import { Plugin, PluginKey } from "@milkdown/kit/prose/state"; import { ReplaceStep } from "@milkdown/kit/prose/transform"; import { $prose } from "@milkdown/kit/utils"; -import { hasActiveSourceProjection, leafdownSourceProjectionPluginKey } from "./sourceProjection"; +import { + hasActiveSourceProjection, + leafdownSourceProjectionPluginKey, + SOURCE_PROJECTION_ENTRY_SUPPRESSION_META, +} from "./sourceProjection"; const ATTENTION_CHARACTERS = "*_~"; +// A tilde is left out. GFM reads a strikethrough only where the closing run matches the opening +// one, so an unequal tilde run spells nothing to pair, and the input rule Leafdown owns for it +// already holds a run literal until the author closes it. +const LITERAL_PAIRING_CHARACTERS = "*_"; const MAXIMUM_RUN_LENGTH = 2; +const MAXIMUM_LITERAL_PAIR_LENGTH = 3; const UNICODE_PUNCTUATION_PATTERN = /[\p{P}\p{S}]/u; +// The preset input rules refuse a run that follows a word character, a colon, or a slash, which +// keeps a delimiter inside a word or a URL literal. The guard is kept, less the underscore that +// `\w` counts as a word character and CommonMark counts as punctuation. +const LITERAL_PAIRING_GUARD_PATTERN = /[\p{L}\p{N}:/]/u; const PAIRED_META = "leafdownAttentionPaired"; export const leafdownAttentionPairingPluginKey = new PluginKey( @@ -34,25 +47,70 @@ interface DelimiterRun { interface AttentionPair { closing: DelimiterRun; - markName: string; + markNames: readonly string[]; opening: DelimiterRun; } -const readMarkName = (character: string, length: number) => { +interface LiteralAttentionPair extends AttentionPair { + closingRunEnd: number; + isOpenToGrowth: boolean; +} + +interface TypedContext { + blockFrom: number; + blockTo: number; + child: InlineChild; + children: readonly InlineChild[]; + index: number; + run: DelimiterRun; +} + +const readMarkNames = (character: string, length: number): readonly string[] => { if (character === "~") { - return "strike_through"; + return ["strike_through"]; } - return length === 1 ? "emphasis" : "strong"; + if (length === 1) { + return ["emphasis"]; + } + + return length === 2 ? ["strong"] : ["emphasis", "strong"]; }; +const isFlankingWhitespace = (character: string | undefined) => + character === undefined || /\s/u.test(character); + +const isFlankingPunctuation = (character: string | undefined) => + character !== undefined && UNICODE_PUNCTUATION_PATTERN.test(character); + // A run pairs only where it flanks its content, and the run answered for here always faces a mark, // whose own delimiters are punctuation. Both CommonMark tests collapse to the character on the // outer side under that neighbour: a run cannot open after a letter or a digit and cannot close // before one. The intraword rule `_` adds falls out of the same neighbour and needs no separate // test. const isOuterNeighbourFlanking = (character: string | undefined) => - character === undefined || /\s/u.test(character) || UNICODE_PUNCTUATION_PATTERN.test(character); + isFlankingWhitespace(character) || isFlankingPunctuation(character); + +// A run facing its own text has no such neighbour to collapse the tests into, so it is measured +// against the characters on both of its sides the way CommonMark states them. +const isLeftFlanking = (before: string | undefined, after: string | undefined) => + !isFlankingWhitespace(after) && + (!isFlankingPunctuation(after) || isFlankingWhitespace(before) || isFlankingPunctuation(before)); + +const isRightFlanking = (before: string | undefined, after: string | undefined) => + !isFlankingWhitespace(before) && + (!isFlankingPunctuation(before) || isFlankingWhitespace(after) || isFlankingPunctuation(after)); + +const canOpenRun = (character: string, before: string | undefined, after: string | undefined) => + isLeftFlanking(before, after) && + (character !== "_" || !isRightFlanking(before, after) || isFlankingPunctuation(before)); + +// A run the author is still typing against has to be unambiguously a closer. CommonMark lets a +// run that flanks on both sides close as well as open, and one closed against a letter reads as +// the opener of a construct the author has not finished, so pairing it would build the wrong one +// around delimiters they are still adding to. +const canCloseRun = (before: string | undefined, after: string | undefined) => + isRightFlanking(before, after) && !isLeftFlanking(before, after); const readInlineChildren = (parent: ProseMirrorNode, start: number) => { const children: InlineChild[] = []; @@ -151,10 +209,10 @@ const carriesMark = (state: EditorState, from: number, to: number, markName: str return carries; }; -const findAttentionPair = ( +const readTypedContext = ( state: EditorState, { character, position }: TypedDelimiter, -): AttentionPair | null => { +): TypedContext | null => { if (position >= state.doc.content.size) { return null; } @@ -179,15 +237,29 @@ const findAttentionPair = ( return null; } - const typed = readTypedRun(child, position, character); - const length = typed.to - typed.from; + return { + blockFrom, + blockTo, + child, + children, + index, + run: readTypedRun(child, position, character), + }; +}; + +const findAttentionPair = ( + state: EditorState, + { blockFrom, blockTo, child, children, index, run }: TypedContext, + character: string, +): AttentionPair | null => { + const length = run.to - run.from; if (length > MAXIMUM_RUN_LENGTH) { return null; } for (const direction of [-1, 1]) { - const facesMark = direction < 0 ? typed.from === child.from : typed.to === child.to; + const facesMark = direction < 0 ? run.from === child.from : run.to === child.to; const counterpart = facesMark ? findCounterpart(children, index, character, length, direction) : null; @@ -196,25 +268,84 @@ const findAttentionPair = ( continue; } - const opening = direction < 0 ? counterpart : typed; - const closing = direction < 0 ? typed : counterpart; - const markName = readMarkName(character, length); + const opening = direction < 0 ? counterpart : run; + const closing = direction < 0 ? run : counterpart; + const markNames = readMarkNames(character, length); if ( !isOuterNeighbourFlanking(readCharacterAt(state, opening.from - 1, blockFrom, blockTo)) || !isOuterNeighbourFlanking(readCharacterAt(state, closing.to, blockFrom, blockTo)) || - state.schema.marks[markName] === undefined || - carriesMark(state, opening.to, closing.from, markName) + markNames.some((markName) => state.schema.marks[markName] === undefined) || + markNames.some((markName) => carriesMark(state, opening.to, closing.from, markName)) ) { continue; } - return { closing, markName, opening }; + return { closing, markNames, opening }; } return null; }; +// The literal counterpart of the pass above: the run the author closed faces its own text rather +// than a mark, so the opening run is the one the same text holds. CommonMark pairs as many +// delimiters as the shorter run spells and leaves the surplus literal, which is why an unequal run +// pairs at all. +// +// A closing run shorter than its opening one is not settled yet, because the next character the +// author types may extend it and pair a longer run. Such a pair is reported as open to growth and +// is applied once the caret leaves its end. +const findLiteralAttentionPair = ( + state: EditorState, + { blockFrom, blockTo, child, run }: TypedContext, + character: string, +): LiteralAttentionPair | null => { + if (!LITERAL_PAIRING_CHARACTERS.includes(character)) { + return null; + } + + const text = child.node.text ?? ""; + const contentEnd = run.from - child.from; + let contentStart = contentEnd; + + while (contentStart > 0 && text[contentStart - 1] !== character) { + contentStart -= 1; + } + + if (contentStart === 0 || contentStart === contentEnd) { + return null; + } + + let openingStart = contentStart; + + while (openingStart > 0 && text[openingStart - 1] === character) { + openingStart -= 1; + } + + const length = Math.min(contentStart - openingStart, run.to - run.from); + const markNames = readMarkNames(character, length); + const beforeOpening = readCharacterAt(state, child.from + openingStart - 1, blockFrom, blockTo); + const afterClosing = readCharacterAt(state, run.to, blockFrom, blockTo); + + if ( + length > MAXIMUM_LITERAL_PAIR_LENGTH || + (beforeOpening !== undefined && LITERAL_PAIRING_GUARD_PATTERN.test(beforeOpening)) || + !canOpenRun(character, beforeOpening, text[contentStart]) || + !canCloseRun(text[contentEnd - 1], afterClosing) || + markNames.some((markName) => state.schema.marks[markName] === undefined) + ) { + return null; + } + + return { + closing: { from: run.from, to: run.from + length }, + closingRunEnd: run.to, + isOpenToGrowth: contentStart - openingStart > run.to - run.from, + markNames, + opening: { from: child.from + contentStart - length, to: child.from + contentStart }, + }; +}; + const readInsertedDelimiter = (transaction: Transaction): TypedDelimiter | null => { for (const step of transaction.steps) { if (!(step instanceof ReplaceStep) || step.from !== step.to) { @@ -244,6 +375,11 @@ const readInsertedDelimiter = (transaction: Transaction): TypedDelimiter | null // The typed character is held until the document settles: a source projection covering the run // commits in a later dispatch, so the mark the run pairs across does not exist yet in the cycle // the character arrives in. +// A pair still open to growth waits for the caret to leave the run it closed, because the author +// may extend that run and pair a longer one. +const isCaretAt = (state: EditorState, position: number) => + state.selection.empty && state.selection.head === position; + const applyTypedDelimiter = ( transaction: Transaction, typed: TypedDelimiter | null, @@ -288,23 +424,47 @@ export const createLeafdownAttentionPairingPlugin = () => return null; } - const pair = findAttentionPair(state, typed); + const context = readTypedContext(state, typed); + const markedPair = context && findAttentionPair(state, context, typed.character); + const literalPair = + context && !markedPair + ? findLiteralAttentionPair(state, context, typed.character) + : null; + + if (literalPair?.isOpenToGrowth && isCaretAt(state, literalPair.closingRunEnd)) { + return null; + } + + const pair = markedPair ?? literalPair; const transaction = state.tr.setMeta(PAIRED_META, true); if (!pair) { return transaction; } - const { closing, markName, opening } = pair; + const { closing, markNames, opening } = pair; const length = opening.to - opening.from; - const mark = state.schema.marks[markName].create( - markName === "strike_through" ? null : { marker: typed.character }, - ); + transaction.delete(closing.from, closing.to).delete(opening.from, opening.to); + + for (const markName of markNames) { + transaction.addMark( + opening.from, + closing.from - length, + state.schema.marks[markName].create( + markName === "strike_through" ? null : { marker: typed.character }, + ), + ); + } + + // The caret ends up against the span the pair just built, which reads the same as a + // caret moved there. Both the marks it would inherit and the projection it would open + // take the author's next character into the construct they just closed, so the pair + // stands as the object it produced until they go back to it. An input rule leaves the + // same caret and is answered the same way. return transaction - .delete(closing.from, closing.to) - .delete(opening.from, opening.to) - .addMark(opening.from, closing.from - length, mark); + .setMeta(SOURCE_PROJECTION_ENTRY_SUPPRESSION_META, true) + .setStoredMarks([]); }, }), );