diff --git a/SDK.md b/SDK.md new file mode 100644 index 0000000..408c9a6 --- /dev/null +++ b/SDK.md @@ -0,0 +1,179 @@ +# crossref-utils (library API) + +Developer-facing docs for the **in-memory** Crossref library after the planned monorepo split. See [`plan-sdk.md`](./plan-sdk.md) for the implementation plan. + +```ts +import { + DoiBatch, + journalXml, + journalArticleFromMyst, + preprintXml, + preprintFromMyst, + conferenceXml, + conferencePaperFromMyst, + databaseXml, + datasetFromMyst, + abstractFromMdast, + generateDoi, + suggestDois, + validateDeposit, +} from 'crossref-utils'; +``` + +CLI (filesystem, myst-cli, prompts) lives in a separate package (`crossref-cli`) and depends on this library. Prefer importing **`crossref-utils` only** from serverless / headless code. + +## Architecture + +``` +┌─────────────────────────────────────────────────────────┐ +│ Callers (serverless, CLI, future tools) │ +└───────────────┬─────────────────────────┬───────────────┘ + │ │ + ▼ ▼ +┌───────────────────────────┐ ┌─────────────────────────┐ +│ Adapters (format → DTO) │ │ Crossref core │ +│ *FromMyst today │ │ types + *Xml │ +│ *FromOther later │──▶│ DoiBatch, dates, … │ +└───────────────────────────┘ │ abstractFromMdast │ + │ generateDoi / suggest │ + │ validateDeposit │ + └─────────────────────────┘ +``` + +**Important separation:** MyST (or any other) *content processing* stays outside the Crossref core. Adapters only map already-structured frontmatter (and helpers like `abstractFromMdast` for processed mdast) into Crossref shapes. That leaves room for other X→Crossref parsers without bloating the core. + +There is **no** separate `buildDeposit` facade or new deposit DTO layer — use the existing Crossref types + `*Xml`, and/or `*FromMyst`. + +--- + +## Inputs (two existing paths) + +### 1. Crossref DTOs → `*Xml` + +Use types from this package (`JournalArticle`, `Preprint`, `ConferenceOptions`, `DatasetMetadata`, `DoiBatchOptions`, …) and builders: + +- `journalXml` / `journalArticleXml` / … +- `conferenceXml` / `conferencePaperXml` +- `preprintXml` +- `databaseXml` / `datasetXml` +- `DoiBatch` → `.toXml()` + +Suitable when the caller already thinks in Crossref terms, or after any future non-MyST adapter. + +### 2. MyST frontmatter → `*FromMyst` (adapter) + +Pass `myst-frontmatter` objects (in memory) into `journalArticleFromMyst`, `preprintFromMyst`, `conferencePaperFromMyst`, `datasetFromMyst`, contributor helpers, etc. + +- No filesystem; no myst-cli `Session` required for project loading. +- Logging via a small **logger** interface (not a full MyST session). +- **DOI `resource` URLs are not defaulted to Curvenote** — supply `doi_data` explicitly or use an injectable resolver where provided (e.g. dataset path today). + +Abstracts are **not** produced from markdown here. Build abstract JATS via `abstractFromMdast` (below), then pass the resulting element/string into the paper options / wire through the adapter as implemented. + +--- + +## `abstractFromMdast` + +```ts +function abstractFromMdast(mdast: GenericParent): Element; // or string — finalize in implementation +``` + +Caller supplies **processed MyST mdast** for the abstract part (from myst-cli or an equivalent pipeline). The library applies the same light transforms as today’s CLI (xrefs→links, cites→text, newlines→spaces), serializes with `myst-to-jats`, and wraps as `jats:abstract`. + +Out of scope for this helper: loading projects, `parseMyst`, extracting parts from a full document. + +```ts +const abstract = abstractFromMdast(abstractMdast); +const article = journalArticleFromMyst(logger, frontmatter, citations, abstract); +``` + +--- + +## DOI helpers + +```ts +function generateDoi(prefix: string): string; +function suggestDois(count: number, prefix: string): string[]; +``` + +- `prefix` is a **numeric DOI prefix** (e.g. `10.62329`). No built-in Curvenote/org alias map in this library — pass prefixes from your config. +- `generateDoi` → `{prefix}/{4 letters}{4 digits}` (unambiguous alphabet). +- `suggestDois` → `count` candidates for a UI or review step. + +**Always human-review** generated DOIs before submitting to Crossref (avoid accidental slur-like or awkward strings). Interactive checkbox selection remains a CLI/caller concern, not part of this package. + +--- + +## `validateDeposit` + +```ts +function validateDeposit(xml: string): Promise; + +type ValidationResult = { + ok: boolean; + errors: { message: string; path?: string; line?: number; column?: number }[]; +}; +``` + +Validates deposit XML **in process** against Crossref schema **5.3.1** (bundled assets; no `xmllint`, no temp-file requirement for the happy path). Usable from serverless. + +Independent of builders — validate XML from `DoiBatch.toXml()` or any other source. + +If a pure-JS XSD engine cannot be maintained, the API still exists with a documented backend (e.g. external service); library callers should not depend on shelling to `xmllint`. + +--- + +## Typical headless flow + +```ts +import { + DoiBatch, + preprintFromMyst, + abstractFromMdast, + generateDoi, + validateDeposit, +} from 'crossref-utils'; + +const doi = generateDoi(process.env.DOI_PREFIX!); // review before use +const abstract = abstractFromMdast(abstractMdast); + +const body = preprintFromMyst(logger, mystFrontmatter, citations, abstract); +// ensure doi_data.resource is set for your host — no Curvenote default + +const batch = new DoiBatch( + { + id: crypto.randomUUID(), + depositor: { name: 'Example', email: 'deposits@example.org' }, + }, + body, +); + +const xml = batch.toXml(); +const { ok, errors } = await validateDeposit(xml); +if (!ok) throw new Error(errors.map((e) => e.message).join('\n')); +``` + +Multi-article journal/conference deposits: build venue/issue XML with existing helpers, map each item with `*FromMyst` or `*Xml`, assemble body, wrap in `DoiBatch`. + +--- + +## What belongs in `crossref-cli` (not this library) + +- Path discovery, reading `myst.yml` / pages from disk +- myst-cli `Session`, `getFileContent`, part extraction from projects +- `parseMyst` for frontmatter abstract strings +- inquirer prompts (deposit type, depositor, DOI checkbox selection) +- Writing DOIs back into config files + +The CLI should call into `crossref-utils` for XML build, abstract mdast→JATS, DOI string generation, and validation. + +--- + +## Package relationship + +| Package | Role | +|---------|------| +| `crossref-utils` | In-memory Crossref core + MyST adapter + validate | +| `crossref-cli` | `crossref` binary; FS + interactive workflows | + +Root import of `crossref-utils` is the supported library surface (no `/sdk` subpath required). diff --git a/plan-sdk.md b/plan-sdk.md new file mode 100644 index 0000000..0fcecd9 --- /dev/null +++ b/plan-sdk.md @@ -0,0 +1,140 @@ +# Plan: Package split for headless Crossref utils + +Revised implementation plan after [PR #27 review](https://github.com/continuous-foundation/crossref-utils/pull/27) (Franklin Koch). Aligns with [`SDK.md`](./SDK.md). + +## Goals + +1. **Split the repo into two packages** (JATS-style monorepo): + - `packages/crossref-utils` — lightweight, in-memory library (the “SDK”) + - `packages/crossref-cli` — filesystem, myst-cli session, inquirer; depends on `-utils` +2. **Keep existing DTOs and APIs** — Crossref types + `*Xml` builders, and MyST frontmatter + `*FromMyst` adapters. No new `DepositInput` / `buildDeposit` facade. +3. **Maintain Crossref vs content-format separation** — core is format-agnostic; MyST is one adapter; future `*FromOther` helpers can map other frontmatter/layouts into Crossref DTOs without growing the core. +4. **Ship in-process XML validation** on `crossref-utils` (replace or supersede CLI-only `xmllint` for library callers). +5. **Remove Curvenote-specific hardcoding** from the library (resource URL defaults, DOI prefix alias map in code). + +## Non-goals + +- New Crossref-oriented deposit DTO layer / single `buildDeposit` entrypoint +- Shipping myst-cli / `parseMyst` / markdown pipelines inside `crossref-utils` +- Interactive DOI checkbox UI inside `-utils` (stays CLI or caller-owned) +- Changing Crossref schema version (stay on 5.3.1) + +## Approach (Franklin’s three steps) + +> Gut check: easier than a long facade plan — an initial package split gets ~90% of the way there. + +1. Tear the single package into `-cli` (FS + interactivity) and `-utils` (everything else). Keep interfaces mostly unchanged. +2. Massage details: decouple abstract *extraction* (CLI / upstream MyST) from light mdast→JATS transforms (utils); swap MyST `Session` for a logger interface on adapters; injectable DOI resource resolution. +3. Land TS-native (or otherwise in-process) XSD validation on `-utils`; retire reliance on `xmllint` for the library API. + +--- + +## Target layout + +``` +packages/ + crossref-utils/ # published as crossref-utils + src/ + # Crossref core + batch.ts, types.ts, dates.ts, contributors.ts, funding.ts, … + journal.ts, conference.ts, preprint.ts, dataset.ts + abstract.ts # mdast → jats:abstract (transforms + myst-to-jats) + doi.ts # generateDoi, suggestDois (prefix always passed in) + validate.ts # in-process XSD validate + fromMyst/ # adapter layer (not “core”) + … *FromMyst helpers, Session→logger refactors + crossref-cli/ # published as crossref-cli (bin: crossref) + src/ + deposit.ts, generate.ts, parse.ts, validate.ts (CLI wrappers) + # FS discovery, inquirer, myst-cli load/extract, write-back to myst.yml +``` + +Published names can stay `crossref-utils` for the library; CLI package name TBD (`crossref-cli` vs keeping a single npm name that re-exports — decide at implement time to minimize breakages). + +--- + +## Workstreams + +### 1. Monorepo split (first, high leverage) + +- Introduce workspace (`packages/*`) following continuous-foundation JATS / similar monorepos. +- Move existing library modules → `crossref-utils`; CLI (`src/cli/**`, bin) → `crossref-cli`. +- `-cli` depends on `-utils`; `-utils` must not depend on commander/inquirer/myst-cli FS workflows. +- Keep `myst-frontmatter` / `myst-to-jats` / light myst types only where needed (adapters + abstract helper). +- Green existing tests; CLI smoke still works via workspace link. + +### 2. Core vs MyST adapter boundary + +**Core (`crossref-utils`):** Crossref DTOs (`types.ts`), `*Xml`, `DoiBatch`, contributors/dates/funding XML, DOI helpers, validate, `abstractFromMdast`. + +**Adapter (`crossref-utils/fromMyst` or equivalent exports):** `*FromMyst` — maps `myst-frontmatter` → Crossref DTOs / elements. Document as the MyST adapter, not the only input path. + +**Future:** other X→Crossref adapters (same package or later packages) map into core DTOs + `*Xml`. Do not fold foreign pipelines into core. + +Refactor `*FromMyst` / `fundrefFromMyst` to take a **logger** (or `Pick`) instead of a full myst `Session`. + +### 3. Abstracts: mdast in, JATS out + +Agreed direction (Franklin option 2 + prior spike): + +- Upstream (CLI or serverless caller) owns myst processing and supplies **processed abstract mdast**. +- `-utils` exposes **`abstractFromMdast(mdast)`** — FS-free transforms + `myst-to-jats` + `jats:abstract` wrap (logic lifted from today’s CLI). +- CLI: split `depositArticleFromSource` — extract/load with myst-cli in CLI; call `abstractFromMdast` from utils. +- Do **not** ship `parseMyst` in `-utils` (avoids pulling full MyST into the library). +- Plain-text-only abstract (lossy option 3) is out of scope as the primary path. + +### 4. DOI helpers & Curvenote cleanup + +- **`generateDoi(prefix: string)`** — keep; numeric prefix **always passed in** (no `curvenote` → `10.62329` map in library code). +- **`suggestDois(count, prefix)`** — non-interactive candidates for UIs; callers must still human-review before Crossref submit (slur / pronounceability concern). +- Prefix aliases (if needed) live in **caller config** or CLI only — not hardcoded in `-utils`. +- Remove hardcoded `https://doi.curvenote.com/...` from `*FromMyst` / helpers; require explicit `doi_data.resource` or an injectable resolver callback on adapters. +- Bonus (not SDK-specific): strip other Curvenote-only defaults as found. + +### 5. Validation on `crossref-utils` (must land) + +- Expose e.g. `validateDeposit(xml: string): Promise` from `-utils`. +- Spike for a **maintained pure JS/Node** XSD validator usable in serverless (no native addon if possible). +- Bundle Crossref 5.3.1 schema assets with the package (or load from package files without network). +- If in-process XSD proves blocked: document fallback (external validation service) but still ship a library `validate*` API contract — do not leave validation CLI/`xmllint`-only. +- CLI can call the utils validator; `xmllint` path may remain as optional local fallback during transition. + +### 6. Docs & release + +- Rewrite [`SDK.md`](./SDK.md) to describe `crossref-utils` as the public library (core + MyST adapter), not a `/sdk` subpath facade. +- Update root README for monorepo / package install (`crossref-utils` vs CLI). +- Changesets per package; note breaking changes (Curvenote defaults removed, Session→logger, package layout). + +--- + +## Suggested order + +1. Monorepo scaffolding + move files (utils vs cli) +2. Logger refactor + Curvenote resource/prefix cleanup +3. Extract `abstractFromMdast`; slim CLI deposit path +4. `generateDoi` / `suggestDois` (prefix required); CLI uses them +5. Validation spike → implement `validateDeposit` on utils +6. Docs (`SDK.md`, README) + changesets + +## Test plan + +- [ ] Workspace build/publish layout for both packages +- [ ] Existing unit tests pass under `crossref-utils` +- [ ] CLI deposit/generate/validate still work against workspace utils +- [ ] `abstractFromMdast` fixture; CLI no longer duplicates transform logic +- [ ] `*FromMyst` works with logger only (no Session) +- [ ] No Curvenote URL/prefix defaults in utils +- [ ] `validateDeposit` catches known-bad deposit XML; known-good passes +- [ ] Serverless-shaped usage: import utils only, no FS + +## Open decisions (small) + +- Exact npm name for the CLI package and whether the current `crossref` bin moves with a major bump. +- Which XSD-in-JS library survives the spike (or external service shape if none does). + +## Explicitly dropped from prior spike plan + +- `crossref-utils/sdk` subpath entry +- New `DepositInput` / `buildDeposit` / `mystToDepositItem` facade types +- Bundling DOI prefix alias constants into the library +- Treating MyST frontmatter JSON as the *only* primary SDK input (both Crossref DTOs and MyST adapters remain)