diff --git a/.changeset/field-mapping-transform-retired.md b/.changeset/field-mapping-transform-retired.md new file mode 100644 index 0000000000..02152086ec --- /dev/null +++ b/.changeset/field-mapping-transform-retired.md @@ -0,0 +1,58 @@ +--- +'@objectstack/spec': major +--- + +按 ADR-0049 enforce-or-remove 退役字段映射的 `transform` 键与整个 `FieldMappingTransform` 联合(#5552) + +`shared/FieldMappingSchema.transform` 由一个五成员判别联合承载(`constant` / `cast` / +`lookup` / `javascript` / `map`),并被两个 `.extend()` 它的 schema 继承 —— +`integration/ConnectorFieldMapping` 与 `data/ExternalFieldMapping`。**五个成员没有一个 +存在执行者**:`fieldMappings` 只在 `packages/spec` 自己的 schema 和测试里被拼写过,四个 +connector 包、automation engine、REST 与 objectui 都不读它,全仓也没有任何代码对 +`transform.type` 分支。这是完整意义上的 declared-but-unenforced(Prime Directive #10), +不只是报单所指的那一个成员。 + +`javascript` 成员是让缺口显形的那一个:它的 `.describe()` 推荐 `dialect: "js"`,而 `js` +方言早在 #3278(ADR-0058 addendum)就退役了 —— 于是文档教的信封写法被枚举直接拒收,唯一 +能通过 parse 的裸字符串又被 `ExpressionInputSchema` 包成 `dialect: 'cel'`,而同一行给出的 +例子 `value.toUpperCase()` 作为 CEL 并不成立。三处互相打架,且三处底下都没有实现。 + +## FROM → TO + +| 你现在写的 | 改成 | +|:---|:---| +| `connector.fieldMappings[].transform: { type: 'cast', targetType: 'string' }` | 删除该键 | +| `connector.fieldMappings[].transform: { type: 'javascript', expression: '…' }` | 删除该键 | +| `externalLookup.fieldMappings[].transform: { … }` | 删除该键 | + +**一句话修复:删掉 `transform` 键。** 没有等价替换成员 —— L3 connector 的字段映射只做 +`source` → `target` 的搬运,从来没有做过值变换。真正要做值变换的地方有两个,都是活的: + +- **导入映射** `mapping.fieldMapping[].transform` —— 一个扁平字符串枚举 + (`none`/`constant`/`map`/`split`/`join`/`lookup`,配置放在 `params`),由 REST 导入 + 路径逐行执行。注意它对自己的 `javascript` 值是**直接 400 拒收**(服务端没有沙箱), + 而不是解析通过然后什么都不做。 +- **ETL transformation 步骤**,面向多源、多阶段的复杂变换。 + +存量元数据无需手改:`os migrate meta --from 16` 会自动重写(ADR-0087 D2 conversion +`field-mapping-transform-removed`);`sys_metadata` 里的存量行在 rehydration 时由 +`applyConversionsToStoredItem` 重放同一条转换。直接 parse 会命中 `retiredKey()` 墓碑, +错误信息本身就是上面这段处方。 + +## 退役套件 + +- **Schema**:`shared/mapping.zod.ts` 上 `transform` 改为 `retiredKey()` 墓碑(该 schema + 与两个 extender 都是普通 `z.object`,直接删键会被静默 strip —— 用另一个静默 no-op 替换 + 原来的静默 no-op);`FieldMappingTransformSchema` / `FieldMappingTransform` 两个导出随键 + 一起删除(无其他消费者的值 schema 会被后来者读成一项能力,#3950)。 +- **D2 conversion**:`field-mapping-transform-removed`,`toMajor: 17`, + `retiredFromLoadPath: true`,重写 `connectors[].fieldMappings[]`。 +- **D3 chain step**:接入 `MIGRATIONS_BY_MAJOR[17].conversionIds` 并扩写 rationale。 +- **退役登记**:`RETIRED_KEYS_BY_MAJOR[17]` 收三个键(一个墓碑 → 三处可作者化拼写,因为 + 两个 extender 各自复制了该属性);`RETIRED_DEFS_BY_MAJOR[17]` 收 + `shared/FieldMappingTransform`。这是自 #4659 / #4725 建表以来两张表的首批条目。 +- **生成物**:`authorable-surface.json` 三行转 `[RETIRED]`;`api-surface.json` −2; + `json-schema.manifest.json` −1 def;spec-changes / upgrade-guide / references 重生成。 + +**未受影响**:`ExpressionDialect` 本体、`ExternalLookup.transform`(lookup 级 +request/response 管线,与字段映射无关)、以及上面那个活着的导入映射 `transform`。 diff --git a/content/docs/getting-started/quick-reference.mdx b/content/docs/getting-started/quick-reference.mdx index 1673d11013..f79d95d514 100644 --- a/content/docs/getting-started/quick-reference.mdx +++ b/content/docs/getting-started/quick-reference.mdx @@ -213,7 +213,7 @@ Common utilities used across all protocols. | **[Expression](/docs/references/shared/expression)** | `expression.zod.ts` | Expression, ExpressionInput | CEL expression values and inputs | | **[HTTP](/docs/references/shared/http)** | `http.zod.ts` | HttpRequest, HttpMethod, CorsConfig | HTTP utilities | | **[Identifiers](/docs/references/shared/identifiers)** | `identifiers.zod.ts` | SystemIdentifier, SnakeCaseIdentifier | Standard ID patterns | -| **[Mapping](/docs/references/shared/mapping)** | `mapping.zod.ts` | FieldMapping, FieldMappingTransform | Field mapping utilities | +| **[Mapping](/docs/references/shared/mapping)** | `mapping.zod.ts` | FieldMapping | Field mapping utilities | | **[Connector Auth](/docs/references/shared/connector-auth)** | `connector-auth.zod.ts` | ConnectorAuthConfig | Connector auth patterns | ## QA Protocol (1 schema) diff --git a/content/docs/references/data/external-lookup.mdx b/content/docs/references/data/external-lookup.mdx index 646b1bbab1..60bdca78b0 100644 --- a/content/docs/references/data/external-lookup.mdx +++ b/content/docs/references/data/external-lookup.mdx @@ -84,10 +84,10 @@ const result = ExternalDataSourceSchema.parse(data); | :--- | :--- | :--- | :--- | | **source** | `string` | ✅ | Source field name | | **target** | `string` | ✅ | Target field name | -| **transform** | `{ type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }` | optional | Transformation to apply | +| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically. | | **defaultValue** | `any` | optional | Default if source is null/undefined | | **type** | `string` | optional | Field type | -| **readonly** | `boolean` | optional | Read-only field | +| **readonly** | `boolean` | ✅ | Read-only field | --- @@ -100,14 +100,14 @@ const result = ExternalDataSourceSchema.parse(data); | :--- | :--- | :--- | :--- | | **fieldName** | `string` | ✅ | Field name | | **dataSource** | `{ id: string; name: string; type: Enum<'odata' \| 'rest-api' \| 'graphql' \| 'custom'>; endpoint: string; … }` | ✅ | External data source | -| **query** | `{ endpoint: string; method?: Enum<'GET' \| 'POST'>; parameters?: Record }` | ✅ | Query configuration | -| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }; defaultValue?: any; … }[]` | ✅ | Field mappings | -| **caching** | `{ enabled?: boolean; ttl?: number; strategy?: Enum<'lru' \| 'lfu' \| 'ttl'> }` | optional | Caching configuration | -| **fallback** | `{ enabled?: boolean; defaultValue?: any; showError?: boolean }` | optional | Fallback configuration | +| **query** | `{ endpoint: string; method: Enum<'GET' \| 'POST'>; parameters?: Record }` | ✅ | Query configuration | +| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; type?: string; … }[]` | ✅ | Field mappings | +| **caching** | `{ enabled: boolean; ttl: number; strategy: Enum<'lru' \| 'lfu' \| 'ttl'> }` | optional | Caching configuration | +| **fallback** | `{ enabled: boolean; defaultValue?: any; showError: boolean }` | optional | Fallback configuration | | **rateLimit** | `{ requestsPerSecond: number; burstSize?: number }` | optional | Rate limiting | -| **retry** | `{ maxRetries?: number; initialDelayMs?: number; maxDelayMs?: number; backoffMultiplier?: number; … }` | optional | Retry configuration with exponential backoff | +| **retry** | `{ maxRetries: number; initialDelayMs: number; maxDelayMs: number; backoffMultiplier: number; … }` | optional | Retry configuration with exponential backoff | | **transform** | `{ request?: object; response?: object }` | optional | Request/response transformation pipeline | -| **pagination** | `{ type?: Enum<'offset' \| 'cursor' \| 'page'>; pageSize?: number; maxPages?: number }` | optional | Pagination configuration for external data | +| **pagination** | `{ type: Enum<'offset' \| 'cursor' \| 'page'>; pageSize: number; maxPages?: number }` | optional | Pagination configuration for external data | --- diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 157c7c1d89..d324715fdc 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,6 +1,6 @@ --- title: Protocol Reference -description: Every schema published by @objectstack/spec — 1610 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1609 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -29,11 +29,11 @@ counts are sums of the rows they head. Regenerate with | [Kernel Protocol](/docs/references/kernel) | 31 | 187 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. | | [Qa Protocol](/docs/references/qa) | 1 | 8 | Declarative test suites — scenarios, steps, actions and assertions. | | [Security Protocol](/docs/references/security) | 5 | 27 | Permission sets, row-level security, sharing rules, tenancy posture. | -| [Shared Protocol](/docs/references/shared) | 8 | 32 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. | +| [Shared Protocol](/docs/references/shared) | 8 | 31 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. | | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | | [System Protocol](/docs/references/system) | 37 | 295 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 17 | 155 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **201** | **1610** | 14 protocol modules | +| **Total** | **201** | **1609** | 14 protocol modules | --- @@ -285,7 +285,7 @@ Permission sets, row-level security, sharing rules, tenancy posture. ## Shared Protocol -**Source:** `packages/spec/src/shared/` · **Import:** `@objectstack/spec/shared` · **8 pages, 32 schemas** +**Source:** `packages/spec/src/shared/` · **Import:** `@objectstack/spec/shared` · **8 pages, 31 schemas** Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. @@ -296,7 +296,7 @@ Primitives used across every protocol — identifiers, HTTP, expressions, error | [`expression.zod.ts`](/docs/references/shared/expression) | `CronExpressionInput`, `Expression`, `ExpressionDialect`, `ExpressionInput`, `ExpressionMeta`, `Predicate`, `PredicateInput`, `TemplateExpressionInput` | | [`http.zod.ts`](/docs/references/shared/http) | `CorsConfig`, `HttpMethod`, `HttpMethodSubset`, `HttpRequest`, `RateLimitConfig`, `StaticMount` | | [`identifiers.zod.ts`](/docs/references/shared/identifiers) | `EventName`, `SnakeCaseIdentifier`, `SystemIdentifier` | -| [`mapping.zod.ts`](/docs/references/shared/mapping) | `FieldMapping`, `FieldMappingTransform` | +| [`mapping.zod.ts`](/docs/references/shared/mapping) | `FieldMapping` | | [`metadata-types.zod.ts`](/docs/references/shared/metadata-types) | `BaseMetadataRecord`, `MetadataFormat` | | [`protection.zod.ts`](/docs/references/shared/protection) | `Protection` | diff --git a/content/docs/references/integration/connector.mdx b/content/docs/references/integration/connector.mdx index 92acd71b7c..229a500d9e 100644 --- a/content/docs/references/integration/connector.mdx +++ b/content/docs/references/integration/connector.mdx @@ -179,7 +179,7 @@ Circuit breaker configuration | **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record; … }[]` | optional | | | **triggers** | `{ key: string; label: string; description?: string; type: Enum<'polling' \| 'webhook'>; … }[]` | optional | Trigger definitions (not yet enforced — never read at registration; see #3197) | | **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration | -| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }; defaultValue?: any; … }[]` | optional | Field mapping rules | +| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; dataType?: Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>; … }[]` | optional | Field mapping rules | | **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) | | **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. | | **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration | @@ -249,11 +249,11 @@ Standard error category | :--- | :--- | :--- | :--- | | **source** | `string` | ✅ | Source field name | | **target** | `string` | ✅ | Target field name | -| **transform** | `{ type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }` | optional | Transformation to apply | +| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically. | | **defaultValue** | `any` | optional | Default if source is null/undefined | | **dataType** | `Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>` | optional | Target data type | -| **required** | `boolean` | optional | Field is required | -| **syncMode** | `Enum<'read_only' \| 'write_only' \| 'bidirectional'>` | optional | Sync mode | +| **required** | `boolean` | ✅ | Field is required | +| **syncMode** | `Enum<'read_only' \| 'write_only' \| 'bidirectional'>` | ✅ | Sync mode | --- @@ -481,7 +481,7 @@ Connector type | **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record; … }[]` | optional | | | **triggers** | `{ key: string; label: string; description?: string; type: Enum<'polling' \| 'webhook'>; … }[]` | optional | Trigger definitions (not yet enforced — never read at registration; see #3197) | | **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration | -| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }; defaultValue?: any; … }[]` | optional | Field mapping rules | +| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; dataType?: Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>; … }[]` | optional | Field mapping rules | | **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) | | **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. | | **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration | diff --git a/content/docs/references/shared/mapping.mdx b/content/docs/references/shared/mapping.mdx index 33c35d1f93..d5ef0ceeed 100644 --- a/content/docs/references/shared/mapping.mdx +++ b/content/docs/references/shared/mapping.mdx @@ -7,16 +7,14 @@ description: Mapping protocol schemas Base Field Mapping Protocol -Shared by: ETL, Connector, External Lookup +Shared by: Connector, External Lookup This module provides the canonical field mapping schema used across -ObjectStack for data transformation and synchronization. +ObjectStack for data synchronization. **Use Cases:** -- ETL pipelines (data/mapping.zod.ts) - - Integration connectors (integration/connector.zod.ts) - External lookups (data/external-lookup.zod.ts) @@ -35,7 +33,7 @@ target: 'user_id', ``` -@example With transformation +@example With a fallback for missing source values ```typescript @@ -45,126 +43,64 @@ source: 'user_name', target: 'name', -transform: \{ type: 'cast', targetType: 'string' \}, - defaultValue: 'Unknown' \}; ``` - -**Source:** `packages/spec/src/shared/mapping.zod.ts` - +## What is NOT here any more: `transform` (#5552, protocol 17) -## TypeScript Usage +This schema used to carry a `transform` key typed by a five-member -```typescript -import { FieldMappingSchema, FieldMappingTransformSchema } from '@objectstack/spec/shared'; -import type { FieldMapping, FieldMappingTransform } from '@objectstack/spec/shared'; +discriminated union — `constant` / `cast` / `lookup` / `javascript` / `map`. -// Validate data -const result = FieldMappingSchema.parse(data); -``` +No runtime ever executed one of the five, so the whole union was retired ---- +under ADR-0049 enforce-or-remove; the tombstone below carries the -## FieldMapping +prescription, and the measurement behind it is written up on the -### Properties +`field-mapping-transform-removed` conversion in `src/conversions/registry.ts`. -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **source** | `string` | ✅ | Source field name | -| **target** | `string` | ✅ | Target field name | -| **transform** | `{ type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record }` | optional | Transformation to apply | -| **defaultValue** | `any` | optional | Default if source is null/undefined | +**Where transforms actually run:** `[data/mapping.zod.ts](/docs/references/data/mapping)`'s +`ImportFieldMappingSchema.transform` — a flat string enum steering a `params` ---- - -## FieldMappingTransform - -### Union Options - -This schema accepts one of the following structures: - -#### Option 1 +bag, applied row by row by the REST import path and recorded live, key by -Set a constant value +key, in `packages/spec/liveness/mapping.json`. Same word, opposite -**Type:** `constant` +disposition: that one runs, and rejects its own `javascript` value with a 400 -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `'constant'` | ✅ | | -| **value** | `any` | ✅ | Constant value to use | +rather than pretending to. ---- - -#### Option 2 - -Cast to a specific data type - -**Type:** `cast` - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `'cast'` | ✅ | | -| **targetType** | `Enum<'string' \| 'number' \| 'boolean' \| 'date'>` | ✅ | Target data type | - ---- - -#### Option 3 - -Lookup value from another table - -**Type:** `lookup` - -### Properties - -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `'lookup'` | ✅ | | -| **table** | `string` | ✅ | Lookup table name | -| **keyField** | `string` | ✅ | Field to match on | -| **valueField** | `string` | ✅ | Field to retrieve | - ---- - -#### Option 4 - -Custom JavaScript transformation + +**Source:** `packages/spec/src/shared/mapping.zod.ts` + -**Type:** `javascript` +## TypeScript Usage -### Properties +```typescript +import { FieldMappingSchema } from '@objectstack/spec/shared'; +import type { FieldMapping } from '@objectstack/spec/shared'; -| Property | Type | Required | Description | -| :--- | :--- | :--- | :--- | -| **type** | `'javascript'` | ✅ | | -| **expression** | `string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }` | ✅ | JS expression (dialect="js" recommended). e.g. value.toUpperCase() | +// Validate data +const result = FieldMappingSchema.parse(data); +``` --- -#### Option 5 - -Map values using a dictionary - -**Type:** `map` +## FieldMapping ### Properties | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **type** | `'map'` | ✅ | | -| **mappings** | `Record` | ✅ | Value mappings (e.g., `{"Active": "active"}`) | - ---- +| **source** | `string` | ✅ | Source field name | +| **target** | `string` | ✅ | Target field name | +| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically. | +| **defaultValue** | `any` | optional | Default if source is null/undefined | --- diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md index c360ecd3c5..45de432c7f 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts.md @@ -268,5 +268,5 @@ directory rather than per file. | `integration/` | 10 | | `kernel/` | 319 | | `qa/` | 6 | -| `shared/` | 25 | +| `shared/` | 20 | | `system/` | 366 | diff --git a/docs/protocol-upgrade-guide.md b/docs/protocol-upgrade-guide.md index ca9e0d30bf..61835e3cff 100644 --- a/docs/protocol-upgrade-guide.md +++ b/docs/protocol-upgrade-guide.md @@ -212,6 +212,8 @@ The same enforce-or-remove reading reaches the storage contract: `IStorageServic Finally it retires the two inert `IndexSchema` keys, `indexes[].type` and `indexes[].partial` (#5248, #4943). Neither ever had a DDL consumer: `SqlDriver.syncDeclaredIndexes` creates declared indexes through knex's `table.index()` / `table.unique()`, and the drift differ's `DeclaredIndexInput` carries only `name`/`fields`/`unique`/`nullSafeColumns` — so an authored `type` selected no access method and an authored `partial` produced a FULL index with its predicate discarded. `partial` was the more damaging of the two because it read as a correctness control: the platform's own `sys_metadata` declared it for overlay uniqueness, and what the declaration alone materialized was an unrestricted unique index (the active-row scoping is delivered by a runtime migration, `metadata-protocol`'s `ensureOverlayIndex`, not by the key). `type` was the louder: its `.default('btree')` put an inert knob into every parse output, so it read as live configuration — the ADR-0078 no-silently-inert shape. Remove was chosen over enforce (maintainer ruling, 2026-08-06): enforcing needs per-dialect algorithm mapping (`gin`/`gist` Postgres-only, `fulltext` MySQL-family), raw-SQL `CREATE INDEX … WHERE` on the dialects that have partial indexes at all (MySQL does not), and a redesign of how `isSyncReproducibleIndex` excludes partial indexes from incremental sync — design cost for a capability nothing has asked for. Both are lossless deletes: no DDL changes, because no DDL ever depended on them. Drift detection is untouched — the `partial` flag it consumes is parsed back out of the database's OWN `CREATE INDEX` DDL and never came from this key. +It also retires the field-mapping `transform` key and the whole five-member `FieldMappingTransform` union behind it (#5552): `constant` / `cast` / `lookup` / `javascript` / `map`, declared on `shared/FieldMapping` and inherited by `integration/ConnectorFieldMapping` and `data/ExternalFieldMapping`. Nothing ever executed one. `fieldMappings` is spelled only inside `packages/spec` itself — the connector packages, the automation engine, REST and objectui never read it, and no code anywhere switches on `transform.type` — so all five members were declared-but-unenforced together, not just the one that got the bug filed. That one is the sharpest evidence though: `javascript`'s `.describe()` recommended the dialect `js`, which `ExpressionDialect` retired at #3278 (ADR-0058 addendum), so the envelope the documentation taught was rejected by the enum; the only spelling that parsed was the bare string, which `ExpressionInputSchema` wraps as `cel`; and the CEL that resulted could not evaluate the `value.toUpperCase()` the same line offered as its example. Three surfaces disagreeing about a capability with no implementation under any of them. Fixing the sentence alone was rejected (maintainer, 2026-08-06) as gilding a member that cannot run. The key is tombstoned rather than deleted because the schema and both extenders are plain `z.object`s and `ConnectorSchema.parse` is a live receiver, so a bare deletion would strip silently. What is NOT affected, despite the shared word: the import mapping's `mapping.fieldMapping[].transform`, a flat string enum applied row by row by the REST import path and live in the liveness ledger — including its own `javascript` value, which that path rejects with a 400 rather than pretending to run. + ### Mechanical (applied for you) | Conversion | Surface | Change | Load window | @@ -257,6 +259,7 @@ Finally it retires the two inert `IndexSchema` keys, `indexes[].type` and `index | `object-enable-trash-mru-removed` | `object.enable.trash / object.enable.mru` | object capability flags 'enable.trash'/'enable.mru' removed (#3207, #2377 close-out — no recycle bin and no MRU tracking ever ran; both default-true flags gated nothing) | retired — `migrate meta` only | | `hook-body-crypto-hash-removed` | `hook.body.capabilities / action.body.capabilities` | script-body capability token 'crypto.hash' removed (#4391 — the sandbox never installed ctx.crypto.hash, so the token granted a call that always threw; the CLI inferred it too) | retired — `migrate meta` only | | `connector-rate-limit-config-removed` | `connector.rateLimitConfig` | connector key 'rateLimitConfig' removed (#4911 — no outbound rate-limiting engine exists; the runtime's only token bucket limits INBOUND requests, so every knob here was inert while reading like a configured cap. The whole ConnectorRateLimitConfig shape went with it) | retired — `migrate meta` only | +| `field-mapping-transform-removed` | `connector.fieldMappings[].transform / externalLookup.fieldMappings[].transform` | field-mapping key 'transform' removed (#5552 — the whole five-member FieldMappingTransform union went with it: no runtime ever executed constant/cast/lookup/javascript/map, and the javascript member advertised dialect="js", retired in #3278. The enforced transform pipeline is the import mapping's string-enum `mapping.fieldMapping[].transform`, which is unaffected) | retired — `migrate meta` only | | `theme-inert-token-scales-removed` | `theme.typography.fontSize / theme.typography.fontWeight / theme.typography.lineHeight / theme.typography.letterSpacing / theme.typography.fontFamily.heading / theme.typography.fontFamily.mono / theme.animation / theme.zIndex` | theme keys 'typography.fontSize'/'fontWeight'/'lineHeight'/'letterSpacing', 'typography.fontFamily.heading'/'mono', 'animation' and 'zIndex' removed (#5021, ADR-0049 — the engine emitted --font-size-*, --font-weight-*, --line-height-*, --letter-spacing-*, --duration-*, --timing-*, --z-*, --font-heading and --font-mono faithfully, and no first-party component or stylesheet has ever read one. Re-declare any variable you actually consume under customVars, which emits it verbatim) | retired — `migrate meta` only | | `page-header-subtitle-alias` | `page.component.page-header.description` | page-header component prop 'description' → 'subtitle' (objectui#3226 — the `subtitle ?? description` fallback retires) | live — protocol 17 loader accepts the old shape | | `object-index-type-partial-removed` | `object.indexes[].type / object.indexes[].partial` | object index keys 'indexes[].type'/'indexes[].partial' removed (#5248, #4943 — no driver ever read either: the index method is the dialect's choice and a partial index is built by a database-layer migration, not declared) | retired — `migrate meta` only | diff --git a/packages/qa/dogfood/test/expression-conformance.ledger.ts b/packages/qa/dogfood/test/expression-conformance.ledger.ts index dd128cd5bc..c1853f2d47 100644 --- a/packages/qa/dogfood/test/expression-conformance.ledger.ts +++ b/packages/qa/dogfood/test/expression-conformance.ledger.ts @@ -80,15 +80,20 @@ export const EXPRESSION_SURFACE: ExprSurface[] = [ }, { id: 'cel-formula', - summary: 'computed / formula field + mapping expressions', + summary: 'computed / formula field expressions', dialect: 'cel', mode: 'interpret', state: 'enforced', failPolicy: 'fail-soft-log', enforcement: '@objectstack/formula celEngine (interpret)', // kernel/feature.zod.ts:expression was covered here until the orphaned // FeatureFlagSchema module was removed (zero runtime consumers once its // capabilities-descriptor home went, #3605). + // shared/mapping.zod.ts:expression went the same way at #5552: it was the + // `javascript` member of FieldMappingTransform, and the whole five-member + // union was retired under ADR-0049 (no runtime executed any of them). The + // surface it named no longer exists in source, so the cover is deleted + // rather than re-pointed — and the summary drops "+ mapping expressions" + // with it, since nothing on that side is left to interpret. covers: [ 'data/field.zod.ts:expression', - 'shared/mapping.zod.ts:expression', ], }, { diff --git a/packages/spec/api-surface/shared.json b/packages/spec/api-surface/shared.json index 74ee70c9d4..327ad021cb 100644 --- a/packages/spec/api-surface/shared.json +++ b/packages/spec/api-surface/shared.json @@ -28,8 +28,6 @@ "F (const)", "FieldMapping (type)", "FieldMappingSchema (const)", - "FieldMappingTransform (type)", - "FieldMappingTransformSchema (const)", "FieldName (type)", "FieldNameSchema (const)", "FlowName (type)", diff --git a/packages/spec/authorable-surface/data.json b/packages/spec/authorable-surface/data.json index b8ae4be0a4..98317abc7a 100644 --- a/packages/spec/authorable-surface/data.json +++ b/packages/spec/authorable-surface/data.json @@ -334,7 +334,7 @@ "data/ExternalFieldMapping:readonly", "data/ExternalFieldMapping:source", "data/ExternalFieldMapping:target", - "data/ExternalFieldMapping:transform", + "data/ExternalFieldMapping:transform [RETIRED]", "data/ExternalFieldMapping:type", "data/ExternalLookup:caching", "data/ExternalLookup:dataSource", diff --git a/packages/spec/authorable-surface/integration.json b/packages/spec/authorable-surface/integration.json index 249b17d5b8..06042c028a 100644 --- a/packages/spec/authorable-surface/integration.json +++ b/packages/spec/authorable-surface/integration.json @@ -42,7 +42,7 @@ "integration/ConnectorFieldMapping:source", "integration/ConnectorFieldMapping:syncMode", "integration/ConnectorFieldMapping:target", - "integration/ConnectorFieldMapping:transform", + "integration/ConnectorFieldMapping:transform [RETIRED]", "integration/ConnectorHealth:circuitBreaker", "integration/ConnectorHealth:healthCheck", "integration/ConnectorInstanceAPIKeyAuth:credentialRef", diff --git a/packages/spec/authorable-surface/shared.json b/packages/spec/authorable-surface/shared.json index 0342b5d23c..bfbb05d77f 100644 --- a/packages/spec/authorable-surface/shared.json +++ b/packages/spec/authorable-surface/shared.json @@ -20,7 +20,7 @@ "shared/FieldMapping:defaultValue", "shared/FieldMapping:source", "shared/FieldMapping:target", - "shared/FieldMapping:transform", + "shared/FieldMapping:transform [RETIRED]", "shared/HttpRequest:body", "shared/HttpRequest:headers", "shared/HttpRequest:method", diff --git a/packages/spec/docs/SYNC_ARCHITECTURE.md b/packages/spec/docs/SYNC_ARCHITECTURE.md index 81899408cb..e82cb91c4f 100644 --- a/packages/spec/docs/SYNC_ARCHITECTURE.md +++ b/packages/spec/docs/SYNC_ARCHITECTURE.md @@ -275,14 +275,15 @@ const sapConnector: ConnectorInput = { source: 'order_value', target: 'order_total', dataType: 'number', - // `transform.type` is a discriminated union with exactly five members: - // `constant` / `cast` / `lookup` / `javascript` / `map`. The bare string - // below is `ExpressionInput` shorthand — the schema wraps it into an - // `{ dialect, source }` envelope on parse. - transform: { - type: 'javascript', - expression: 'value / 100' // Convert cents to dollars - }, + // (`transform` sat here until #5552 retired it, together with the whole + // five-member `FieldMappingTransform` union — `constant` / `cast` / + // `lookup` / `javascript` / `map`. None of the five ever had an executor: + // an L3 connector mapping moves a value from `source` to `target`, and + // nothing anywhere read the transform. The `javascript` member is what + // made the gap visible — it recommended the retired `js` dialect + // (#3278), so the only spelling that parsed was a bare string, which + // means CEL. Value conversion belongs on a surface that runs it: the L2 + // import mapping's own `transform`, or an ETL transformation step.) syncMode: 'bidirectional' } ], diff --git a/packages/spec/json-schema.manifest/shared.json b/packages/spec/json-schema.manifest/shared.json index c6edb58096..0554b3fbbb 100644 --- a/packages/spec/json-schema.manifest/shared.json +++ b/packages/spec/json-schema.manifest/shared.json @@ -12,7 +12,6 @@ "shared/ExpressionInput", "shared/ExpressionMeta", "shared/FieldMapping", - "shared/FieldMappingTransform", "shared/FieldName", "shared/FlowName", "shared/HttpMethod", diff --git a/packages/spec/spec-changes.json b/packages/spec/spec-changes.json index f8c137492f..7e69a91d3b 100644 --- a/packages/spec/spec-changes.json +++ b/packages/spec/spec-changes.json @@ -314,6 +314,12 @@ "conversionId": "connector-rate-limit-config-removed", "toMajor": 17 }, + { + "surface": "connector.fieldMappings[].transform / externalLookup.fieldMappings[].transform", + "to": "field-mapping key 'transform' removed (#5552 — the whole five-member FieldMappingTransform union went with it: no runtime ever executed constant/cast/lookup/javascript/map, and the javascript member advertised dialect=\"js\", retired in #3278. The enforced transform pipeline is the import mapping's string-enum `mapping.fieldMapping[].transform`, which is unaffected)", + "conversionId": "field-mapping-transform-removed", + "toMajor": 17 + }, { "surface": "theme.typography.fontSize / theme.typography.fontWeight / theme.typography.lineHeight / theme.typography.letterSpacing / theme.typography.fontFamily.heading / theme.typography.fontFamily.mono / theme.animation / theme.zIndex", "to": "theme keys 'typography.fontSize'/'fontWeight'/'lineHeight'/'letterSpacing', 'typography.fontFamily.heading'/'mono', 'animation' and 'zIndex' removed (#5021, ADR-0049 — the engine emitted --font-size-*, --font-weight-*, --line-height-*, --letter-spacing-*, --duration-*, --timing-*, --z-*, --font-heading and --font-mono faithfully, and no first-party component or stylesheet has ever read one. Re-declare any variable you actually consume under customVars, which emits it verbatim)", @@ -1042,6 +1048,12 @@ "conversionId": "connector-rate-limit-config-removed", "toMajor": 17 }, + { + "surface": "connector.fieldMappings[].transform / externalLookup.fieldMappings[].transform", + "to": "field-mapping key 'transform' removed (#5552 — the whole five-member FieldMappingTransform union went with it: no runtime ever executed constant/cast/lookup/javascript/map, and the javascript member advertised dialect=\"js\", retired in #3278. The enforced transform pipeline is the import mapping's string-enum `mapping.fieldMapping[].transform`, which is unaffected)", + "conversionId": "field-mapping-transform-removed", + "toMajor": 17 + }, { "surface": "theme.typography.fontSize / theme.typography.fontWeight / theme.typography.lineHeight / theme.typography.letterSpacing / theme.typography.fontFamily.heading / theme.typography.fontFamily.mono / theme.animation / theme.zIndex", "to": "theme keys 'typography.fontSize'/'fontWeight'/'lineHeight'/'letterSpacing', 'typography.fontFamily.heading'/'mono', 'animation' and 'zIndex' removed (#5021, ADR-0049 — the engine emitted --font-size-*, --font-weight-*, --line-height-*, --letter-spacing-*, --duration-*, --timing-*, --z-*, --font-heading and --font-mono faithfully, and no first-party component or stylesheet has ever read one. Re-declare any variable you actually consume under customVars, which emits it verbatim)", diff --git a/packages/spec/src/automation/etl-author-shape.test.ts b/packages/spec/src/automation/etl-author-shape.test.ts index 47f7dd0a47..0bfd75e64c 100644 --- a/packages/spec/src/automation/etl-author-shape.test.ts +++ b/packages/spec/src/automation/etl-author-shape.test.ts @@ -119,7 +119,8 @@ describe('[#4963] SYNC_ARCHITECTURE.md pipeline examples compile', () => { // bare `...`, which is not TypeScript; one full `sapConnector` example) and // compiles the third. When this pin was written that example reported four // diagnostics, three of them keys or values the schema REJECTS (`sourceField` - // / `targetField` for `source` / `target`, `transform.type: 'custom'`, + // / `targetField` for `source` / `target`, `transform.type: 'custom'` — + // which #5552 then made moot by retiring `transform` outright — // `webhooks[].retryPolicy`); those are fixed in the document. The fourth was // `Connector` being `z.infer` — this issue's twin on a file whose migration // surface is NOT empty — and it is solved there by annotating the example diff --git a/packages/spec/src/conversions/registry.ts b/packages/spec/src/conversions/registry.ts index 4ab6cc9303..9a921eaccd 100644 --- a/packages/spec/src/conversions/registry.ts +++ b/packages/spec/src/conversions/registry.ts @@ -4128,6 +4128,144 @@ const connectorRateLimitConfigRemoved: MetadataConversion = { }, }; +/** + * `fieldMappings[].transform` — five declared transforms, zero engines (#5552, + * ADR-0049). + * + * `shared/FieldMappingSchema.transform` was typed by a five-member + * discriminated union (`FieldMappingTransformSchema`: `constant` / `cast` / + * `lookup` / `javascript` / `map`), inherited by both schemas that extend it — + * `integration/ConnectorFieldMapping` and `data/ExternalFieldMapping`. The + * whole union goes; the key is tombstoned on the base, which retires all three + * authorable spellings in one edit. + * + * What was measured, and against what (2026-08-06, `origin/main` @ efedd28): + * + * - **Declarative**: `transform: { type: … }` in a field-mapping position is + * authored NOWHERE outside `packages/spec`'s own tests — not in `examples/`, + * not in `skills/`, not in objectui. The showcase's one mapping + * (`examples/app-showcase/src/data/mappings/index.ts`) writes `transform: + * 'map'` — a bare STRING, which is the *other* schema (see below). + * - **Parse**: reachable. `AutomationEngine.registerConnector` / + * `registerDegradedConnector` run `ConnectorSchema.parse`, which walks + * `fieldMappings[]`. So the tombstone's prescription has a live receiver, + * which is why this is a `retiredKey()` and not a bare deletion. + * - **Execution**: none. `fieldMappings` is spelled only inside + * `packages/spec` — the four connector packages, the automation engine, REST + * and objectui never read it, and nothing anywhere switches on + * `transform.type`. Falsification control for the scan: the member-specific + * words are findable in the tree (`targetType` 70 hits, `keyField` 66, + * `valueField` 260) — the scanner works, the consumers are absent. + * - **cloud**: NOT verified. This session has no read access to + * `objectstack-ai/cloud`, and GitHub code search does not index it (a control + * query scoped to that repo returns zero with `incomplete_results`). Recorded + * as unverified rather than claimed clean — the #5540 disposition. + * + * The `javascript` member is the one that got the defect filed (#5552): its + * `.describe()` recommended `dialect="js"` while `ExpressionDialect` retired + * `js` in #3278 (ADR-0058 addendum), so the envelope the doc taught was + * rejected by the enum, the only spelling that parsed was the bare string — + * which `ExpressionInputSchema` wraps as `dialect: 'cel'` — and the example it + * offered (`value.toUpperCase()`) is not valid CEL. The maintainer ruling + * (2026-08-06) rejected fixing the sentence alone as "gilding a member that + * cannot run" and ordered enforce-or-remove on the measurement; the + * measurement came back dead for all five. + * + * NOT touched, and the reason the word `transform` survives elsewhere: + * `data/mapping.zod.ts`'s `ImportFieldMappingSchema.transform` is a different + * declaration under the same name — a flat string enum steering `params`, + * applied row by row by `packages/rest/src/import-mapping.ts:115-167` and + * recorded live in `packages/spec/liveness/mapping.json`. It is also the + * counter-example that makes this retirement's shape clear: its own + * `javascript` value is rejected with a 400 because no server sandbox exists — + * implement-or-reject-loudly, rather than a member that parses and evaporates. + * + * Scope of the rewrite: `connectors[].fieldMappings[]` is the only stored + * source that can carry the key. `ExternalLookupSchema` is not referenced by + * any stack collection or metadata type, so there is no external-lookup + * document for the walker to visit; its authorable key is retired by the same + * tombstone and needs no transform. + * + * `retiredFromLoadPath`: the key claims a transformation that never ran, the + * `connector-rate-limit-config-removed` shape exactly. Absorbing it silently at + * load would let an author keep believing their values were being cast, mapped + * or looked up. The entry exists so stored rows replay clean + * (`applyConversionsToStoredItem`) and `os migrate meta --from 16` rewrites + * author sources; live parses hit the tombstone. + */ +const fieldMappingTransformRemoved: MetadataConversion = { + id: 'field-mapping-transform-removed', + toMajor: 17, + retiredFromLoadPath: true, + surface: 'connector.fieldMappings[].transform / externalLookup.fieldMappings[].transform', + summary: + "field-mapping key 'transform' removed (#5552 — the whole five-member " + + 'FieldMappingTransform union went with it: no runtime ever executed constant/cast/' + + 'lookup/javascript/map, and the javascript member advertised dialect="js", retired ' + + "in #3278. The enforced transform pipeline is the import mapping's string-enum " + + '`mapping.fieldMapping[].transform`, which is unaffected)', + apply(stack, emit) { + return mapCollection(stack, 'connectors', (c, path) => { + const mappings = c.fieldMappings; + if (!Array.isArray(mappings)) return c; + let changed = false; + const next = mappings.map((m, i) => { + if (!m || typeof m !== 'object' || Array.isArray(m)) return m; + const stripped = stripKeys( + m as Record, + ['transform'], + emit, + `${path}.fieldMappings[${i}]`, + ); + if (stripped !== m) changed = true; + return stripped; + }); + return changed ? { ...c, fieldMappings: next } : c; + }); + }, + fixture: { + before: { + connectors: [ + { + name: 'sap_erp', + label: 'SAP ERP', + type: 'saas', + fieldMappings: [ + // The member #5552 was filed about: `dialect: 'js'` was never + // parseable, so the only authored form is the bare string — which + // silently meant CEL. + { source: 'order_value', target: 'order_total', transform: { type: 'javascript', expression: 'value / 100' } }, + // A second member, to show the notice is per mapping entry and not + // per union member. + { source: 'Status', target: 'status', transform: { type: 'map', mappings: { Active: 'active' } } }, + // A mapping that never authored the key keeps its identity — the + // copy-on-write contract `stripKeys` / `mapCollection` are built on. + { source: 'E-mail', target: 'email', defaultValue: '' }, + ], + }, + // A connector with no field mappings at all is untouched. + { name: 'crm_sync', label: 'CRM Sync', type: 'saas' }, + ], + }, + after: { + connectors: [ + { + name: 'sap_erp', + label: 'SAP ERP', + type: 'saas', + fieldMappings: [ + { source: 'order_value', target: 'order_total' }, + { source: 'Status', target: 'status' }, + { source: 'E-mail', target: 'email', defaultValue: '' }, + ], + }, + { name: 'crm_sync', label: 'CRM Sync', type: 'saas' }, + ], + }, + expectedNotices: 2, + }, +}; + /** * The nine theme token groups that were EMITTED and read by nobody (#5021, * ADR-0049). @@ -4430,6 +4568,7 @@ export const CONVERSIONS_BY_MAJOR: Readonly { - const message = render(results.get('transform-custom')!); - expect(message).toContain('"custom"'); - // The five real members, named, so that adding or removing one is a - // decision that surfaces here rather than silently widening the doc's claim. - for (const member of ['constant', 'cast', 'lookup', 'javascript', 'map']) { - expect(message, `the union must still offer ${member}`).toContain(member); - } + // ⚠ Measured, not assumed — and it is the one place the two tombstone + // channels are NOT equally good. `retiredKey()` is `z.never().optional()`, so + // its `z.input` type is `undefined`, and tsc reports the assignment failure + // against that type: "Type '{ … }' is not assignable to type 'undefined'". + // The compile channel therefore REFUSES the key but does not NAME it, while + // the parse channel (the `[#5515]`/`[#5552]` runtime block below) carries the + // full prescription. Asserting `toContain('transform')` here was the first + // draft and it was simply wrong about the diagnostic text; pinning the real + // shape is what keeps this test honest about which channel says what. + it('[#5552] `transform` is retired — the key no longer type-checks', () => { + // Was: "`custom` is not a member of the transform union", asserting all five + // member names appeared in the message. That assertion cannot be re-spelled + // — the union it enumerated is gone — so it is replaced by the fact that + // survived: the key itself fails to compile. + const message = render(results.get('transform-retired')!); + expect(message).toContain('TS2322'); + expect(message).toContain("not assignable to type 'undefined'"); + }); + + it('[#5552] …and a member that used to be VALID fails identically', () => { + // The guard against a silent restoration: if the union came back, this probe + // would compile and go green, which is the only signal distinguishing "the + // key is retired" from "that one value was never a member". Identical + // diagnostic to the probe above — same key, same refusal, regardless of the + // value's shape. + const message = render(results.get('transform-retired-valid-member')!); + expect(message).toContain('TS2322'); + expect(message).toContain("not assignable to type 'undefined'"); }); it('`webhooks[].retryPolicy` does not exist on the webhook shape', () => { @@ -312,31 +351,45 @@ describe('[#5515] the schema rejects them at RUNTIME too, and how it says so', ( expect(JSON.stringify(result.error!.issues)).not.toContain('sourceField'); }); - it("`transform.type: 'custom'` is a VALUE verdict, and the message lists the five real members", () => { - const result = ConnectorFieldMappingSchema.safeParse({ - source: 'order_value', - target: 'order_total', - transform: { type: 'custom', function: 'value => parseFloat(value) / 100' }, - }); - expect(result.success).toBe(false); - expect(result.error!.issues[0]!.message).toContain("Expected 'constant' | 'cast' | 'lookup' | 'javascript' | 'map'"); + it('[#5552] `transform` is now a KEY verdict carrying the retirement prescription', () => { + // The verdict CHANGED CLASS here, and that is the point worth pinning. + // #5515 measured a VALUE verdict — the key was real, `'custom'` was not a + // member, and the message enumerated the five that were. #5552 retired the + // key, so the same input is now refused one level up, by the key, and every + // member is equally out. Both spellings below get the identical message, + // which is what "the union is gone" means as opposed to "your value was + // wrong". + for (const transform of [ + { type: 'custom', function: 'value => parseFloat(value) / 100' }, + { type: 'javascript', expression: 'value / 100' }, + ]) { + const result = ConnectorFieldMappingSchema.safeParse({ + source: 'order_value', + target: 'order_total', + transform, + }); + expect(result.success).toBe(false); + const issue = result.error!.issues.find((i) => i.path.join('.') === 'transform'); + expect(issue).toBeDefined(); + expect(issue!.message).toMatch(/`FieldMapping\.transform`.*removed.*#5552/s); + // It must point at the transform pipeline that DOES run, not just refuse. + expect(issue!.message).toMatch(/mapping\.fieldMapping\[\]\.transform/s); + } }); - it('the corrected mapping parses, and a bare expression string is wrapped into its envelope', () => { + it('the corrected mapping parses — with the transform dropped, not re-spelled', () => { + // There is no replacement member to move to: the L3 connector surface never + // transformed anything. What the author keeps is the plain source→target + // mapping; what they must move elsewhere is the transformation itself. const parsed = ConnectorFieldMappingSchema.parse({ source: 'order_value', target: 'order_total', dataType: 'number', - transform: { type: 'javascript', expression: 'value / 100' }, syncMode: 'bidirectional', }); - // `ExpressionInputSchema` shorthand: the string the document writes is the - // INPUT, the envelope is what a parse returns. Stated here so the doc's - // one-line form is known to be the schema's own shorthand and not a guess. - expect(parsed.transform).toEqual({ - type: 'javascript', - expression: { dialect: 'cel', source: 'value / 100' }, - }); + expect(parsed).not.toHaveProperty('transform'); + expect(parsed.source).toBe('order_value'); + expect(parsed.dataType).toBe('number'); }); }); diff --git a/packages/spec/src/integration/connector.test.ts b/packages/spec/src/integration/connector.test.ts index b21a008fa6..0479eb44c0 100644 --- a/packages/spec/src/integration/connector.test.ts +++ b/packages/spec/src/integration/connector.test.ts @@ -170,18 +170,22 @@ describe('ConnectorFieldMappingSchema', () => { expect(() => ConnectorFieldMappingSchema.parse(mapping)).not.toThrow(); }); - it('should accept field with transformation', () => { - const mapping = { + // Was `should accept field with transformation`, asserting this exact literal + // parsed and came back as `type: 'javascript'`. Replaced rather than + // re-spelled: #5552 retired the key and the whole union behind it, so there is + // no other member to move the fixture to. Its `value.toUpperCase()` is also + // the string that got the bug filed — `ExpressionInputSchema` wrapped it as + // `dialect: 'cel'`, where that method does not exist. + it('[#5552] rejects a field transformation — the key and its union are retired', () => { + const result = ConnectorFieldMappingSchema.safeParse({ source: 'name', target: 'full_name', - transform: { - type: 'javascript' as const, - expression: 'value.toUpperCase()', - }, - }; - - const parsed = ConnectorFieldMappingSchema.parse(mapping); - expect(parsed.transform?.type).toBe('javascript'); + transform: { type: 'javascript', expression: 'value.toUpperCase()' }, + }); + + expect(result.success).toBe(false); + expect(result.error!.issues[0]!.path.join('.')).toBe('transform'); + expect(result.error!.issues[0]!.message).toMatch(/removed in @objectstack\/spec 17\.0\.0/s); }); it('should use default values', () => { @@ -917,18 +921,17 @@ describe('[#4703] FieldMapping no longer names three declarations', () => { const sharedEntry = await import('../shared/index'); const integrationEntry = await import('./index'); - // Four keys, `transform` a discriminated union, `source`/`target` required. + // Three live keys since #5552 retired `transform`: `source`/`target` + // required, `defaultValue` optional. expect( sharedEntry.FieldMappingSchema.parse({ source: 'FirstName', target: 'first_name', - transform: { type: 'cast', targetType: 'string' }, defaultValue: '', }), ).toEqual({ source: 'FirstName', target: 'first_name', - transform: { type: 'cast', targetType: 'string' }, defaultValue: '', }); @@ -939,36 +942,42 @@ describe('[#4703] FieldMapping no longer names three declarations', () => { ); }); - // ── Difference 1: `transform` is the same key name with mutually - // unparseable value types. The hardest evidence that these are two - // concepts rather than three spellings of one. - it('`transform` means a discriminated union on two sides and a flat enum on the third', async () => { + // ── Difference 1: `transform` is the same key name meaning opposite things. + // Until #5552 that read "a discriminated union on two sides and a flat + // enum on the third", and the object/enum forms were mutually unparseable. + // The retirement did not soften the difference, it sharpened it: on + // shared/integration the key is now RETIRED — the union it named had no + // executor on any of the five members — while on ./data it is the live, + // enforced import pipeline. So the same word is now "gone, with a + // prescription" versus "runs on every imported row", which is the loudest + // the distinction has ever been, and the reason a snippet copied across + // these domains can no longer half-work. + it('`transform` is retired on shared/integration and live on ./data', async () => { const dataEntry = await import('../data/index'); const sharedEntry = await import('../shared/index'); const integrationEntry = await import('./index'); const unionForm = { type: 'cast' as const, targetType: 'string' as const }; - // shared / integration: the object form parses… - expect( - sharedEntry.FieldMappingSchema.parse({ source: 'a', target: 'b', transform: unionForm }) - .transform, - ).toEqual(unionForm); - expect( - integrationEntry.ConnectorFieldMappingSchema.parse({ - source: 'a', - target: 'b', - transform: unionForm, - }).transform, - ).toEqual(unionForm); - // …and the enum form does NOT. + // shared / integration: the object form is refused BY NAME, with the #5552 + // prescription — not stripped, and not a generic "unrecognized key". + for (const schema of [ + sharedEntry.FieldMappingSchema, + integrationEntry.ConnectorFieldMappingSchema, + ]) { + const result = schema.safeParse({ source: 'a', target: 'b', transform: unionForm }); + expect(result.success).toBe(false); + expect(result.error!.issues.some((i) => /#5552/.test(i.message))).toBe(true); + } + // The enum form does not get in either — retired is retired, whatever the + // value's shape. expect( sharedEntry.FieldMappingSchema.safeParse({ source: 'a', target: 'b', transform: 'join' }) .success, ).toBe(false); - // data: exactly the other way round — a bare enum steering a flat `params` - // bag, defaulting to 'none'. + // data: untouched by #5552 — a bare enum steering a flat `params` bag, + // defaulting to 'none', and still rejecting the union form. expect( dataEntry.ImportFieldMappingSchema.parse({ source: 'a', target: 'b' }).transform, ).toBe('none'); diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index bd31a8c26f..aa1e5ba1b9 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -1023,7 +1023,29 @@ const step17: MigrationStep = { + 'incremental sync — design cost for a capability nothing has asked for. Both are lossless ' + 'deletes: no DDL changes, because no DDL ever depended on them. Drift detection is ' + 'untouched — the `partial` flag it consumes is parsed back out of the database\'s OWN ' - + '`CREATE INDEX` DDL and never came from this key.', + + '`CREATE INDEX` DDL and never came from this key.\n\n' + + 'It also retires the field-mapping `transform` key and the whole five-member ' + + '`FieldMappingTransform` union behind it (#5552): `constant` / `cast` / `lookup` / ' + + '`javascript` / `map`, declared on `shared/FieldMapping` and inherited by ' + + '`integration/ConnectorFieldMapping` and `data/ExternalFieldMapping`. Nothing ever ' + + 'executed one. `fieldMappings` is spelled only inside `packages/spec` itself — the ' + + 'connector packages, the automation engine, REST and objectui never read it, and no ' + + 'code anywhere switches on `transform.type` — so all five members were ' + + 'declared-but-unenforced together, not just the one that got the bug filed. That one ' + + 'is the sharpest evidence though: `javascript`\'s `.describe()` recommended the ' + + 'dialect `js`, which `ExpressionDialect` retired at #3278 (ADR-0058 addendum), so the ' + + 'envelope the documentation taught was rejected by the enum; the only spelling that ' + + 'parsed was the bare string, which `ExpressionInputSchema` wraps as `cel`; and the CEL ' + + 'that resulted could not evaluate the `value.toUpperCase()` the same line offered as ' + + 'its example. Three surfaces disagreeing about a capability with no implementation ' + + 'under any of them. Fixing the sentence alone was rejected (maintainer, 2026-08-06) as ' + + 'gilding a member that cannot run. The key is tombstoned rather than deleted because ' + + 'the schema and both extenders are plain `z.object`s and `ConnectorSchema.parse` is a ' + + 'live receiver, so a bare deletion would strip silently. What is NOT affected, despite ' + + 'the shared word: the import mapping\'s `mapping.fieldMapping[].transform`, a flat ' + + 'string enum applied row by row by the REST import path and live in the liveness ' + + 'ledger — including its own `javascript` value, which that path rejects with a 400 ' + + 'rather than pretending to run.', conversionIds: [ 'action-execute-to-target', 'field-conditionalRequired-to-requiredWhen', @@ -1069,6 +1091,7 @@ const step17: MigrationStep = { 'theme-inert-token-scales-removed', 'page-header-subtitle-alias', 'object-index-type-partial-removed', + 'field-mapping-transform-removed', ], semantic: [ { @@ -2101,8 +2124,17 @@ export const MIGRATION_MAJORS: readonly number[] = Object.keys(MIGRATIONS_BY_MAJ * @see scripts/build-schemas.ts — checks (b)/(b2), the only consumers */ export const RETIRED_KEYS_BY_MAJOR: Readonly> = { - // Empty by design at protocol 17: see "Not a backfill of history" above. The - // first entry arrives with the first retirement tombstoned after #4659. + // The first entries since #4659 built this table (#5552). ONE tombstone + // produces THREE keys: `transform` is declared on `shared/FieldMapping` and + // `integration/ConnectorFieldMapping` / `data/ExternalFieldMapping` are + // `.extend()`s of it, so the retired property is copied into all three walked + // shapes and `authorable-surface.json` marks each `[RETIRED]` separately. + // Registered per key, as the gate reads them — nothing radiates from the base. + 17: [ + 'data/ExternalFieldMapping:transform', + 'integration/ConnectorFieldMapping:transform', + 'shared/FieldMapping:transform', + ], }; /** @@ -2178,6 +2210,9 @@ export const RETIRED_KEYS_BY_MAJOR: Readonly> * @see scripts/build-schemas.ts — the manifest deletion gate, the only consumer */ export const RETIRED_DEFS_BY_MAJOR: Readonly> = { - // Empty by design at protocol 17: see "Not a backfill of history" above. The - // first entry arrives with the first whole-schema removal after #4725. + // The first entry since #4725 built this table (#5552). The `transform` key's + // value schema had no other consumer, so it goes with the key rather than + // surviving as an exported union nothing references — an exported schema with + // no consumer reads as a capability to whoever finds it (#3950). + 17: ['shared/FieldMappingTransform'], }; diff --git a/packages/spec/src/shared/mapping.test.ts b/packages/spec/src/shared/mapping.test.ts index ec1004d1b7..2a59fb9320 100644 --- a/packages/spec/src/shared/mapping.test.ts +++ b/packages/spec/src/shared/mapping.test.ts @@ -1,72 +1,7 @@ import { describe, it, expect } from 'vitest'; -import { FieldMappingTransformSchema, FieldMappingSchema } from './mapping.zod'; - -describe('FieldMappingTransformSchema', () => { - it('should accept constant transform', () => { - const result = FieldMappingTransformSchema.parse({ type: 'constant', value: 'hello' }); - expect(result).toEqual({ type: 'constant', value: 'hello' }); - }); - - it('should accept constant transform with any value type', () => { - expect(() => FieldMappingTransformSchema.parse({ type: 'constant', value: 42 })).not.toThrow(); - expect(() => FieldMappingTransformSchema.parse({ type: 'constant', value: null })).not.toThrow(); - expect(() => FieldMappingTransformSchema.parse({ type: 'constant', value: true })).not.toThrow(); - }); - - it('should accept cast transform with valid target types', () => { - const validTypes = ['string', 'number', 'boolean', 'date']; - validTypes.forEach((t) => { - const result = FieldMappingTransformSchema.parse({ type: 'cast', targetType: t }); - expect(result).toEqual({ type: 'cast', targetType: t }); - }); - }); - - it('should reject cast transform with invalid target type', () => { - expect(() => FieldMappingTransformSchema.parse({ type: 'cast', targetType: 'array' })).toThrow(); - }); - - it('should accept lookup transform', () => { - const result = FieldMappingTransformSchema.parse({ - type: 'lookup', - table: 'users', - keyField: 'id', - valueField: 'name', - }); - expect(result).toEqual({ - type: 'lookup', - table: 'users', - keyField: 'id', - valueField: 'name', - }); - }); - - it('should reject lookup transform missing required fields', () => { - expect(() => FieldMappingTransformSchema.parse({ type: 'lookup', table: 'users' })).toThrow(); - }); - - it('should accept javascript transform', () => { - const result = FieldMappingTransformSchema.parse({ - type: 'javascript', - expression: 'value.toUpperCase()', - }); - expect(result).toEqual({ type: 'javascript', expression: { dialect: 'cel', source: 'value.toUpperCase()' } }); - }); - - it('should accept map transform', () => { - const result = FieldMappingTransformSchema.parse({ - type: 'map', - mappings: { Active: 'active', Inactive: 'inactive' }, - }); - expect(result).toEqual({ - type: 'map', - mappings: { Active: 'active', Inactive: 'inactive' }, - }); - }); - - it('should reject unknown transform type', () => { - expect(() => FieldMappingTransformSchema.parse({ type: 'unknown' })).toThrow(); - }); -}); +import { FieldMappingSchema } from './mapping.zod'; +import { ConnectorFieldMappingSchema } from '../integration/connector.zod'; +import { ExternalFieldMappingSchema } from '../data/external-lookup.zod'; describe('FieldMappingSchema', () => { it('should accept minimal valid mapping', () => { @@ -80,15 +15,6 @@ describe('FieldMappingSchema', () => { }); }); - it('should accept mapping with transform', () => { - const result = FieldMappingSchema.parse({ - source: 'user_name', - target: 'name', - transform: { type: 'cast', targetType: 'string' }, - }); - expect(result.transform).toEqual({ type: 'cast', targetType: 'string' }); - }); - it('should accept mapping with defaultValue', () => { const result = FieldMappingSchema.parse({ source: 'user_name', @@ -98,16 +24,14 @@ describe('FieldMappingSchema', () => { expect(result.defaultValue).toBe('Unknown'); }); - it('should accept mapping with all fields', () => { + it('should accept mapping with all live fields', () => { const result = FieldMappingSchema.parse({ source: 'FirstName', target: 'first_name', - transform: { type: 'cast', targetType: 'string' }, defaultValue: '', }); expect(result.source).toBe('FirstName'); expect(result.target).toBe('first_name'); - expect(result.transform).toBeDefined(); expect(result.defaultValue).toBe(''); }); @@ -119,12 +43,78 @@ describe('FieldMappingSchema', () => { expect(() => FieldMappingSchema.parse({ source: 'name' })).toThrow(); }); - it('should have optional transform and defaultValue', () => { + it('should have optional defaultValue', () => { const result = FieldMappingSchema.parse({ source: 'a', target: 'b', }); - expect(result.transform).toBeUndefined(); expect(result.defaultValue).toBeUndefined(); }); }); + +// ============================================================================ +// [#5552] `transform` and the whole FieldMappingTransform union are RETIRED +// ============================================================================ +// +// The pins below replace an entire `describe('FieldMappingTransformSchema')` +// block that asserted all five members parsed. Every one of those cases was +// green for the wrong reason: they proved the union *parsed*, which was never in +// doubt — nothing anywhere ever *executed* one. Re-spelling them was not an +// option and neither was keeping them; the fixture they pinned is exactly what +// was removed, so they are replaced with pins on the surviving behaviour: the +// prescription, and the strip. +// +// The reverse-verification direction, decided before running it: put the union +// and the `transform` key back and these three go RED — the first two because +// no error is raised at all (the parse succeeds and returns the transform), the +// third because `toHaveProperty` finds the key it asserts is gone. That is the +// ordinary direction, not one of the inverted ones, because the tombstone is +// the *only* thing producing these verdicts: there is no schema-level rejection +// underneath it to fall through to (`FieldMappingSchema` is a plain `z.object`, +// so without the tombstone an authored `transform` is either accepted or +// silently stripped — never named). + +describe('[#5552] FieldMapping.transform is retired, and says so', () => { + const RETIRED = { + source: 'order_value', + target: 'order_total', + transform: { type: 'javascript', expression: 'value / 100' }, + }; + + it('the base schema rejects it with the prescription, not a generic error', () => { + const result = FieldMappingSchema.safeParse(RETIRED); + expect(result.success).toBe(false); + // The `s` flag: the guidance spans lines once a reporter wraps it. + expect(result.error!.issues[0]!.message).toMatch( + /`FieldMapping\.transform`.*removed.*17\.0\.0.*#5552/s, + ); + // It must name the live mechanism, not merely refuse: an author who wrote a + // transform wants to know where transforms actually run. + expect(result.error!.issues[0]!.message).toMatch(/mapping\.fieldMapping\[\]\.transform/s); + // And the migration command, since the conversion rewrites sources. + expect(result.error!.issues[0]!.message).toMatch(/os migrate meta --from 16/s); + }); + + it('both extenders inherit the tombstone — one retirement, three authorable spellings', () => { + // `ConnectorFieldMappingSchema` and `ExternalFieldMappingSchema` are + // `.extend()`s of the base, so the retired property is copied into their + // shapes. This is why `RETIRED_KEYS_BY_MAJOR` registers three keys for one + // tombstone; if `.extend()` ever stopped copying it, this goes red. + for (const [name, schema] of [ + ['ConnectorFieldMapping', ConnectorFieldMappingSchema], + ['ExternalFieldMapping', ExternalFieldMappingSchema], + ] as const) { + const result = schema.safeParse(RETIRED); + expect(result.success, `${name} must reject the retired key`).toBe(false); + expect(result.error!.issues.some((i) => i.path.join('.') === 'transform')).toBe(true); + } + }); + + it('a mapping without the key parses and carries no `transform` at all', () => { + // The positive half: the strip path. `not.toHaveProperty` rather than + // `toBeUndefined` — a key present with an undefined value would still let a + // consumer's `'transform' in mapping` check succeed. + const parsed = FieldMappingSchema.parse({ source: 'order_value', target: 'order_total' }); + expect(parsed).not.toHaveProperty('transform'); + }); +}); diff --git a/packages/spec/src/shared/mapping.zod.ts b/packages/spec/src/shared/mapping.zod.ts index 3fa7e1a405..2ac0d26a9c 100644 --- a/packages/spec/src/shared/mapping.zod.ts +++ b/packages/spec/src/shared/mapping.zod.ts @@ -1,21 +1,20 @@ // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. import { z } from 'zod'; -import { ExpressionInputSchema } from './expression.zod'; +import { retiredKey } from './retired-key'; /** * Base Field Mapping Protocol - * - * Shared by: ETL, Connector, External Lookup + * + * Shared by: Connector, External Lookup * * This module provides the canonical field mapping schema used across - * ObjectStack for data transformation and synchronization. + * ObjectStack for data synchronization. * * **Use Cases:** - * - ETL pipelines (data/mapping.zod.ts) * - Integration connectors (integration/connector.zod.ts) * - External lookups (data/external-lookup.zod.ts) - * + * * @example Basic field mapping * ```typescript * const mapping: FieldMapping = { @@ -23,77 +22,49 @@ import { ExpressionInputSchema } from './expression.zod'; * target: 'user_id', * }; * ``` - * - * @example With transformation + * + * @example With a fallback for missing source values * ```typescript * const mapping: FieldMapping = { * source: 'user_name', * target: 'name', - * transform: { type: 'cast', targetType: 'string' }, * defaultValue: 'Unknown' * }; * ``` - */ - -/** - * Field Mapping Transform Schema * - * Defines the transformation to apply to a field value during mapping. - * Implementations can extend this for domain-specific transforms. + * ## What is NOT here any more: `transform` (#5552, protocol 17) * - * Renamed from `TransformTypeSchema` (#4539): its inferred type exported as - * `TransformType`, colliding with the data domain's import-mapping enum of - * the same name under a DIFFERENT shape (config-object union vs string enum) - * — the #4411 dual-source trap. Neither old name had importers outside this - * module in framework/cloud/objectui, so the rename is a clean break. + * This schema used to carry a `transform` key typed by a five-member + * discriminated union — `constant` / `cast` / `lookup` / `javascript` / `map`. + * No runtime ever executed one of the five, so the whole union was retired + * under ADR-0049 enforce-or-remove; the tombstone below carries the + * prescription, and the measurement behind it is written up on the + * `field-mapping-transform-removed` conversion in `src/conversions/registry.ts`. + * + * **Where transforms actually run:** `data/mapping.zod.ts`'s + * `ImportFieldMappingSchema.transform` — a flat string enum steering a `params` + * bag, applied row by row by the REST import path and recorded live, key by + * key, in `packages/spec/liveness/mapping.json`. Same word, opposite + * disposition: that one runs, and rejects its own `javascript` value with a 400 + * rather than pretending to. */ -import { lazySchema } from './lazy-schema'; -export const FieldMappingTransformSchema = lazySchema(() => z.discriminatedUnion('type', [ - z.object({ - type: z.literal('constant'), - value: z.unknown().describe('Constant value to use'), - }).describe('Set a constant value'), - - z.object({ - type: z.literal('cast'), - targetType: z.enum(['string', 'number', 'boolean', 'date']).describe('Target data type'), - }).describe('Cast to a specific data type'), - - z.object({ - type: z.literal('lookup'), - table: z.string().describe('Lookup table name'), - keyField: z.string().describe('Field to match on'), - valueField: z.string().describe('Field to retrieve'), - }).describe('Lookup value from another table'), - - z.object({ - type: z.literal('javascript'), - expression: ExpressionInputSchema.describe('JS expression (dialect="js" recommended). e.g. value.toUpperCase()'), - }).describe('Custom JavaScript transformation'), - - z.object({ - type: z.literal('map'), - mappings: z.record(z.string(), z.unknown()).describe('Value mappings (e.g., {"Active": "active"})'), - }).describe('Map values using a dictionary'), -])); -export type FieldMappingTransform = z.infer; +import { lazySchema } from './lazy-schema'; /** * Field Mapping Schema - * + * * Base schema for mapping fields between source and target systems. - * + * * **NAMING CONVENTION:** * - source: Field name in the source system * - target: Field name in the target system (should be snake_case for ObjectStack) - * + * * @example * ```typescript * { * source: 'FirstName', * target: 'first_name', - * transform: { type: 'cast', targetType: 'string' }, * defaultValue: '' * } * ``` @@ -103,17 +74,31 @@ export const FieldMappingSchema = lazySchema(() => z.object({ * Source field name */ source: z.string().describe('Source field name'), - + /** * Target field name (should be snake_case for ObjectStack) */ target: z.string().describe('Target field name'), - + /** - * Transformation to apply + * REMOVED at protocol 17 (#5552, ADR-0049). See the module TSDoc above for + * the measurement. Tombstoned rather than deleted because this schema and + * both of its extenders are plain `z.object`s: a plain delete would strip the + * key silently, replacing one silent no-op with another. */ - transform: FieldMappingTransformSchema.optional().describe('Transformation to apply'), - + transform: retiredKey( + '`FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and ' + + '`externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 ' + + '(#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it ' + + '(`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed ' + + 'any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect ' + + 'retired in #3278. Delete the key. The transform pipeline that IS enforced is the ' + + "import mapping's: `mapping.fieldMapping[].transform` (a string enum — " + + '`none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), ' + + 'applied by the REST import path, which rejects `javascript` with a 400 rather than ' + + 'pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically.', + ), + /** * Default value if source is null/undefined */ diff --git a/packages/spec/variant-docs.json b/packages/spec/variant-docs.json index b21dfcef77..4b9dc04a09 100644 --- a/packages/spec/variant-docs.json +++ b/packages/spec/variant-docs.json @@ -107,12 +107,6 @@ "exempt": "not-authorable", "reason": "Chosen by the operator when a tenant is provisioned, not written into any tenant's metadata — so no hand-written authoring page owes it a mention. (Was labelled generated-reference-only, though its own reason already said `operator-set`; the generated page at content/docs/references/system/tenant.mdx remains the reference.)" }, - { - "key": "type:cast|constant|javascript|lookup|map", - "label": "sync mapping transform", - "exempt": "generated-reference-only", - "reason": "DOCUMENTATION GAP. `mappings:` is authored on a stack (releases/v12.mdx records `defineMapping`), but no hand-written page covers the transform set — the only prose that names a mapping at all is protocol/kernel/config-resolution.mdx, on a different subject. Lower priority than the connector gap: the transforms are a closed, self-describing set. Bind a page here when one is written." - }, { "key": "kind:action|http|navigate", "label": "settings-manifest handler",