Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .changeset/field-mapping-transform-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
'@objectstack/spec': major
---

按 ADR-0049 enforce-or-remove 退役字段映射的 `transform` 键与整个 `FieldMappingTransform` 联合(#5552)

`shared/FieldMappingSchema.transform` 由一个五成员判别联合承载(`constant` / `cast` /
`lookup` / `javascript` / `map`),并被两个 `.extend()` 它的 schema 继承 ——
`integration/ConnectorFieldMapping` 与 `data/ExternalFieldMapping`。**五个成员没有一个
存在执行者**:`fieldMappings` 只在 `packages/spec` 自己的 schema 和测试里被拼写过,四个
connector 包、automation engine、REST 与 objectui 都不读它,全仓也没有任何代码对
`transform.type` 分支。这是完整意义上的 declared-but-unenforced(Prime Directive #10),
不只是报单所指的那一个成员。

`javascript` 成员是让缺口显形的那一个:它的 `.describe()` 推荐 `dialect: "js"`,而 `js`
方言早在 #3278(ADR-0058 addendum)就退役了 —— 于是文档教的信封写法被枚举直接拒收,唯一
能通过 parse 的裸字符串又被 `ExpressionInputSchema` 包成 `dialect: 'cel'`,而同一行给出的
例子 `value.toUpperCase()` 作为 CEL 并不成立。三处互相打架,且三处底下都没有实现。

## FROM → TO

| 你现在写的 | 改成 |
|:---|:---|
| `connector.fieldMappings[].transform: { type: 'cast', targetType: 'string' }` | 删除该键 |
| `connector.fieldMappings[].transform: { type: 'javascript', expression: '…' }` | 删除该键 |
| `externalLookup.fieldMappings[].transform: { … }` | 删除该键 |

**一句话修复:删掉 `transform` 键。** 没有等价替换成员 —— L3 connector 的字段映射只做
`source` → `target` 的搬运,从来没有做过值变换。真正要做值变换的地方有两个,都是活的:

- **导入映射** `mapping.fieldMapping[].transform` —— 一个扁平字符串枚举
(`none`/`constant`/`map`/`split`/`join`/`lookup`,配置放在 `params`),由 REST 导入
路径逐行执行。注意它对自己的 `javascript` 值是**直接 400 拒收**(服务端没有沙箱),
而不是解析通过然后什么都不做。
- **ETL transformation 步骤**,面向多源、多阶段的复杂变换。

存量元数据无需手改:`os migrate meta --from 16` 会自动重写(ADR-0087 D2 conversion
`field-mapping-transform-removed`);`sys_metadata` 里的存量行在 rehydration 时由
`applyConversionsToStoredItem` 重放同一条转换。直接 parse 会命中 `retiredKey()` 墓碑,
错误信息本身就是上面这段处方。

## 退役套件

- **Schema**:`shared/mapping.zod.ts` 上 `transform` 改为 `retiredKey()` 墓碑(该 schema
与两个 extender 都是普通 `z.object`,直接删键会被静默 strip —— 用另一个静默 no-op 替换
原来的静默 no-op);`FieldMappingTransformSchema` / `FieldMappingTransform` 两个导出随键
一起删除(无其他消费者的值 schema 会被后来者读成一项能力,#3950)。
- **D2 conversion**:`field-mapping-transform-removed`,`toMajor: 17`,
`retiredFromLoadPath: true`,重写 `connectors[].fieldMappings[]`。
- **D3 chain step**:接入 `MIGRATIONS_BY_MAJOR[17].conversionIds` 并扩写 rationale。
- **退役登记**:`RETIRED_KEYS_BY_MAJOR[17]` 收三个键(一个墓碑 → 三处可作者化拼写,因为
两个 extender 各自复制了该属性);`RETIRED_DEFS_BY_MAJOR[17]` 收
`shared/FieldMappingTransform`。这是自 #4659 / #4725 建表以来两张表的首批条目。
- **生成物**:`authorable-surface.json` 三行转 `[RETIRED]`;`api-surface.json` −2;
`json-schema.manifest.json` −1 def;spec-changes / upgrade-guide / references 重生成。

**未受影响**:`ExpressionDialect` 本体、`ExternalLookup.transform`(lookup 级
request/response 管线,与字段映射无关)、以及上面那个活着的导入映射 `transform`。
2 changes: 1 addition & 1 deletion content/docs/getting-started/quick-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ Common utilities used across all protocols.
| **[Expression](/docs/references/shared/expression)** | `expression.zod.ts` | Expression, ExpressionInput | CEL expression values and inputs |
| **[HTTP](/docs/references/shared/http)** | `http.zod.ts` | HttpRequest, HttpMethod, CorsConfig | HTTP utilities |
| **[Identifiers](/docs/references/shared/identifiers)** | `identifiers.zod.ts` | SystemIdentifier, SnakeCaseIdentifier | Standard ID patterns |
| **[Mapping](/docs/references/shared/mapping)** | `mapping.zod.ts` | FieldMapping, FieldMappingTransform | Field mapping utilities |
| **[Mapping](/docs/references/shared/mapping)** | `mapping.zod.ts` | FieldMapping | Field mapping utilities |
| **[Connector Auth](/docs/references/shared/connector-auth)** | `connector-auth.zod.ts` | ConnectorAuthConfig | Connector auth patterns |

## QA Protocol (1 schema)
Expand Down
16 changes: 8 additions & 8 deletions content/docs/references/data/external-lookup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ const result = ExternalDataSourceSchema.parse(data);
| :--- | :--- | :--- | :--- |
| **source** | `string` | ✅ | Source field name |
| **target** | `string` | ✅ | Target field name |
| **transform** | `{ type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }` | optional | Transformation to apply |
| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically. |
| **defaultValue** | `any` | optional | Default if source is null/undefined |
| **type** | `string` | optional | Field type |
| **readonly** | `boolean` | optional | Read-only field |
| **readonly** | `boolean` | | Read-only field |


---
Expand All @@ -100,14 +100,14 @@ const result = ExternalDataSourceSchema.parse(data);
| :--- | :--- | :--- | :--- |
| **fieldName** | `string` | ✅ | Field name |
| **dataSource** | `{ id: string; name: string; type: Enum<'odata' \| 'rest-api' \| 'graphql' \| 'custom'>; endpoint: string; … }` | ✅ | External data source |
| **query** | `{ endpoint: string; method?: Enum<'GET' \| 'POST'>; parameters?: Record<string, any> }` | ✅ | Query configuration |
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | ✅ | Field mappings |
| **caching** | `{ enabled?: boolean; ttl?: number; strategy?: Enum<'lru' \| 'lfu' \| 'ttl'> }` | optional | Caching configuration |
| **fallback** | `{ enabled?: boolean; defaultValue?: any; showError?: boolean }` | optional | Fallback configuration |
| **query** | `{ endpoint: string; method: Enum<'GET' \| 'POST'>; parameters?: Record<string, any> }` | ✅ | Query configuration |
| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; type?: string; … }[]` | ✅ | Field mappings |
| **caching** | `{ enabled: boolean; ttl: number; strategy: Enum<'lru' \| 'lfu' \| 'ttl'> }` | optional | Caching configuration |
| **fallback** | `{ enabled: boolean; defaultValue?: any; showError: boolean }` | optional | Fallback configuration |
| **rateLimit** | `{ requestsPerSecond: number; burstSize?: number }` | optional | Rate limiting |
| **retry** | `{ maxRetries?: number; initialDelayMs?: number; maxDelayMs?: number; backoffMultiplier?: number; … }` | optional | Retry configuration with exponential backoff |
| **retry** | `{ maxRetries: number; initialDelayMs: number; maxDelayMs: number; backoffMultiplier: number; … }` | optional | Retry configuration with exponential backoff |
| **transform** | `{ request?: object; response?: object }` | optional | Request/response transformation pipeline |
| **pagination** | `{ type?: Enum<'offset' \| 'cursor' \| 'page'>; pageSize?: number; maxPages?: number }` | optional | Pagination configuration for external data |
| **pagination** | `{ type: Enum<'offset' \| 'cursor' \| 'page'>; pageSize: number; maxPages?: number }` | optional | Pagination configuration for external data |


---
Expand Down
10 changes: 5 additions & 5 deletions content/docs/references/index.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Protocol Reference
description: Every schema published by @objectstack/spec — 1610 schemas across 14 protocol modules
description: Every schema published by @objectstack/spec — 1609 schemas across 14 protocol modules
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
Expand Down Expand Up @@ -29,11 +29,11 @@ counts are sums of the rows they head. Regenerate with
| [Kernel Protocol](/docs/references/kernel) | 31 | 187 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. |
| [Qa Protocol](/docs/references/qa) | 1 | 8 | Declarative test suites — scenarios, steps, actions and assertions. |
| [Security Protocol](/docs/references/security) | 5 | 27 | Permission sets, row-level security, sharing rules, tenancy posture. |
| [Shared Protocol](/docs/references/shared) | 8 | 32 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. |
| [Shared Protocol](/docs/references/shared) | 8 | 31 | Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums. |
| [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
| [System Protocol](/docs/references/system) | 37 | 295 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
| [UI Protocol](/docs/references/ui) | 17 | 155 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
| **Total** | **201** | **1610** | 14 protocol modules |
| **Total** | **201** | **1609** | 14 protocol modules |

---

Expand Down Expand Up @@ -285,7 +285,7 @@ Permission sets, row-level security, sharing rules, tenancy posture.

## Shared Protocol

**Source:** `packages/spec/src/shared/` · **Import:** `@objectstack/spec/shared` · **8 pages, 32 schemas**
**Source:** `packages/spec/src/shared/` · **Import:** `@objectstack/spec/shared` · **8 pages, 31 schemas**

Primitives used across every protocol — identifiers, HTTP, expressions, error maps, enums.

Expand All @@ -296,7 +296,7 @@ Primitives used across every protocol — identifiers, HTTP, expressions, error
| [`expression.zod.ts`](/docs/references/shared/expression) | `CronExpressionInput`, `Expression`, `ExpressionDialect`, `ExpressionInput`, `ExpressionMeta`, `Predicate`, `PredicateInput`, `TemplateExpressionInput` |
| [`http.zod.ts`](/docs/references/shared/http) | `CorsConfig`, `HttpMethod`, `HttpMethodSubset`, `HttpRequest`, `RateLimitConfig`, `StaticMount` |
| [`identifiers.zod.ts`](/docs/references/shared/identifiers) | `EventName`, `SnakeCaseIdentifier`, `SystemIdentifier` |
| [`mapping.zod.ts`](/docs/references/shared/mapping) | `FieldMapping`, `FieldMappingTransform` |
| [`mapping.zod.ts`](/docs/references/shared/mapping) | `FieldMapping` |
| [`metadata-types.zod.ts`](/docs/references/shared/metadata-types) | `BaseMetadataRecord`, `MetadataFormat` |
| [`protection.zod.ts`](/docs/references/shared/protection) | `Protection` |

Expand Down
10 changes: 5 additions & 5 deletions content/docs/references/integration/connector.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ Circuit breaker configuration
| **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record<string, any>; … }[]` | optional | |
| **triggers** | `{ key: string; label: string; description?: string; type: Enum<'polling' \| 'webhook'>; … }[]` | optional | Trigger definitions (not yet enforced — never read at registration; see #3197) |
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; dataType?: Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>; … }[]` | optional | Field mapping rules |
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
| **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
Expand Down Expand Up @@ -249,11 +249,11 @@ Standard error category
| :--- | :--- | :--- | :--- |
| **source** | `string` | ✅ | Source field name |
| **target** | `string` | ✅ | Target field name |
| **transform** | `{ type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }` | optional | Transformation to apply |
| **transform** | `never` | optional | [REMOVED] `FieldMapping.transform` — authored as `connector.fieldMappings[].transform` and `externalLookup.fieldMappings[].transform` — was removed in @objectstack/spec 17.0.0 (#5552, ADR-0049), and the whole `FieldMappingTransform` union went with it (`constant` / `cast` / `lookup` / `javascript` / `map`) — no runtime ever executed any of the five, and the `javascript` member advertised `dialect: "js"`, a dialect retired in #3278. Delete the key. The transform pipeline that IS enforced is the import mapping's: `mapping.fieldMapping[].transform` (a string enum — `none`/`constant`/`map`/`split`/`join`/`lookup` — with its settings in `params`), applied by the REST import path, which rejects `javascript` with a 400 rather than pretending to run it. Run `os migrate meta --from 16` to rewrite it automatically. |
| **defaultValue** | `any` | optional | Default if source is null/undefined |
| **dataType** | `Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>` | optional | Target data type |
| **required** | `boolean` | optional | Field is required |
| **syncMode** | `Enum<'read_only' \| 'write_only' \| 'bidirectional'>` | optional | Sync mode |
| **required** | `boolean` | | Field is required |
| **syncMode** | `Enum<'read_only' \| 'write_only' \| 'bidirectional'>` | | Sync mode |


---
Expand Down Expand Up @@ -481,7 +481,7 @@ Connector type
| **actions** | `{ key: string; label: string; description?: string; inputSchema?: Record<string, any>; … }[]` | optional | |
| **triggers** | `{ key: string; label: string; description?: string; type: Enum<'polling' \| 'webhook'>; … }[]` | optional | Trigger definitions (not yet enforced — never read at registration; see #3197) |
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
| **fieldMappings** | `{ source: string; target: string; defaultValue?: any; dataType?: Enum<'string' \| 'number' \| 'boolean' \| 'date' \| 'datetime' \| 'json' \| 'array'>; … }[]` | optional | Field mapping rules |
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
| **rateLimitConfig** | `never` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
Expand Down
Loading
Loading