Skip to content

feat(usage): register Cognition/Devin list prices for both devin surfaces - #4466

Merged
lidge-jun merged 1 commit into
devfrom
codex/devin-price-overlays
Sep 13, 2026
Merged

feat(usage): register Cognition/Devin list prices for both devin surfaces#4466
lidge-jun merged 1 commit into
devfrom
codex/devin-price-overlays

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • Registers Cognition/Devin per-model list prices as expected-price overlays for both OAuth surfaces (`devin` and `devin-cli`), which previously resolved every attempt as price_unmatched.
  • Source: the official Devin docs "AI Models" page (docs.devin.ai/desktop/models) embeds a modelCostData table publishing input / cache-read / cache-write / output per model uid, verified 2026-09-13. Self-serve overage and enterprise ACU conversion both bill at these list rates.
  • All 26 rows are verified-derived: the tuples are the vendor's published list rates shown as estimates on a subscription/ACU surface. SWE-2's list rate ($3/$15/$0.30) is identical to Kimi K3's.
  • Time-boxed promos are deliberately not baked in (SWE-2 $0 self-serve through 2026-10-08, 75%-off enterprise through 2026-12-31); they are recorded in the source strings. gemini-3-8-flash is absent from the Devin table and derives from Google's published promotional rate instead.

Verification

  • `bun test tests/usage/usage-cost.test.ts` — 97 pass / 0 fail (focused check; includes the updated 121-key membership assertion and new devin resolve assertions).
  • Repository-wide suite, typecheck, GUI build: NOT RUN (no GUI or runtime-path change; overlay data only).

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed. (Not needed — usage-cost estimate data only; provenance lives in the source strings.)
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (Pricing metadata only; no auth, credential, or workflow changes.)

Summary by CodeRabbit

  • New Features

    • Added pricing information for Devin models, including SWE, GPT, Claude, GLM, Kimi, Gemini, and Grok offerings.
    • Added Devin and Devin CLI pricing overlays for usage cost calculations.
  • Tests

    • Expanded pricing coverage to include the new Devin overlays.
    • Added validation for selected Devin model cost calculations.

…aces

The devin and devin-cli providers had no rows in the generated metadata
bundle or the expected-price overlays, so every attempt resolved
price_unmatched. Cognition publishes per-model list rates in the official
Devin docs modelCostData table (docs.devin.ai/desktop/models, verified
2026-09-13); both self-serve overage and enterprise ACU conversion bill at
those rates. Register them as verified-derived estimates on the
subscription/ACU surface, covering the full liveModels roster for both
provider ids. Time-boxed promos (SWE-2 free self-serve through 2026-10-08,
75% off enterprise through 2026-12-31) are noted in the source strings but
not baked into the tuples. gemini-3-8-flash is absent from the Devin table
and derives from Google's published rate instead.
@lidge-jun
lidge-jun requested a review from Ingwannu as a code owner September 13, 2026 06:00
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-13T06:06:30.404982Z 38c1e24 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the enhancement New feature or request label Sep 13, 2026
@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 4f69fc27-dae0-4aac-985e-b758f5e5efba

📥 Commits

Reviewing files that changed from the base of the PR and between 2df82f4 and 38c1e24.

📒 Files selected for processing (2)
  • src/usage/expected-prices.ts
  • tests/usage/usage-cost.test.ts

📝 Walkthrough

Walkthrough

The change adds verified Cognition/Devin pricing data for devin-cli and devin, updates the expected overlay membership, and verifies representative resolved cost tuples.

Changes

Devin pricing support

Layer / File(s) Summary
Devin pricing catalog
src/usage/expected-prices.ts:146-170, src/usage/expected-prices.ts:333-365
Adds seven Devin Cost4 constants, source notes, and devin-cli/devin overlay rows with verified-derived pricing metadata.
Overlay validation
tests/usage/usage-cost.test.ts:301-302, tests/usage/usage-cost.test.ts:400-428, tests/usage/usage-cost.test.ts:460-469
Updates the expected overlay count, adds Devin membership keys, and checks resolved tuples for three Devin model/provider combinations.

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Feature

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/devin-price-overlays

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun
lidge-jun merged commit 9cbad1b into dev Sep 13, 2026
25 of 26 checks passed
@lidge-jun
lidge-jun deleted the codex/devin-price-overlays branch September 13, 2026 06:01
@lidge-jun

Copy link
Copy Markdown
Owner Author

리뷰 · 우선순위 66 / 80

이 PR은 Cognition/Devin 쪽 사용량(Usage) 추정에서 계속 price_unmatched로 떨어지던 구멍을 메운다. 지금 dev HEAD(2df82f412, #4464 Devin 유닛 문서 마감)에는 devin / devin-cli 두 OAuth 표면이 살아 있고(레지스트리 liveModels: true, ACP는 #4415에서 이미 퇴역), 최근에도 #4445 SWE-2 effort, #4450 headersMs, #4453 PromptCache, #4459 tier-suffix 정리까지 Devin 런타임이 많이 올라왔다. 그런데 src/usage/expected-prices.tsEXPECTED_PRICE_OVERLAYS에는 두 provider id 행이 전혀 없어서, 시도(attempt)가 가격 매칭에 실패하면 추정 비용이 비거나 unmatched로 남는다. 같은 날 패턴의 선례는 #4444(Z.AI GLM list price overlay, verified-derived)이고, 이 PR도 그 길을 그대로 따른다.

구체적으로 docs.devin.ai/desktop/models 의 공식 modelCostData 표(입력/캐시읽기/캐시쓰기/출력, 2026-09-13 확인)에 나온 list rate를 Cost4 상수로 옮기고, devin-cli 13행 + devin 13행 = 26행을 overlay에 등록한다. 테스트는 멤버십을 95→121로 올리고, SWE-2 / swe-1-7-lightning / gpt-5-6-luna 세 건의 resolveMatchedPrice 값을 직접 검증한다. 한시 프로모(SWE-2 셀프서브 $0 ~2026-10-08, 엔터프라이즈 75% 할인 ~2026-12-31 등)는 튜플에 넣지 않고 source 문자열에만 적어 둔다. 구독/ACU 표면이라 status는 전부 verified-derived다. 런타임·GUI·워크플로 변경은 없다. overlay 데이터 + 사용량 테스트만이다.

레지스트리 정적 seed와 비교하면 의도가 분명하다. devin-cli seed는 swe-2, swe-1-7, gpt-5-6-sol, gpt-6-astra, claude-opus-5, claude-fable-5-1, claude-sonnet-5, glm-5-3, kimi-k3, gemini-3-8-flash, grok-4-6 이고, devin seed는 swe-1-7, swe-1-7-lightning, gpt-5-6-sol/luna/terra, claude-opus-4-8, claude-fable-5-1, claude-sonnet-5, glm-5-2, kimi-k2-7, grok-4-5 이다. PR은 여기에 live catalog 드리프트를 대비해 양쪽에 swe-2 / swe-1-6 / swe-1-7-lightning 등을 교차로 더 넣었다. resolveMatchedPrice가 provider id 정확 매칭이라 표면마다 행을 복제하는 것도 #4444 계열과 같다. gemini-3-8-flash는 Devin 표에 없어서 Google 공개 프로모 요금(GEMINI_38_FLASH)으로 파생한 점도 source 문자열에 명시되어 있다.

전체적으로 “Devin을 쓰기 시작한 사람이 Usage에서 가격을 못 보는” 실제 빈칸을, 이미 검증된 overlay 계약으로 채우는 PR이다. 우선순위 66은 그 실용성(높음)과 위험(낮음, 데이터만)의 균형이다. 머지 전에 docs 표 숫자 몇 개만 눈으로 맞추면 충분하다.

src/usage/expected-prices.ts DEVIN_SONNET_5 (대략 164행 근처) - Anthropic 쪽 CLAUDE_SONNET_46(3/15/0.3/3.75)과 다른 Devin 전용 튜플(2/10/0.2/2.5)을 새로 둔 것은 맞다. 다만 docs 표와 한 번 더 대조해 캐시쓰기 2.5가 list column인지 확인이 필요하다.

src/usage/expected-prices.ts DEVIN_KIMI_K27 (대략 166행) - cacheWrite: 0이다. HEAD의 Kimi 계열(KIMI_K27_CODE 등)은 cacheWrite가 0.95인 행이 많다. Devin 표가 정말 0을 찍는지, 아니면 미공개를 0으로 둔 건지 source에 한 줄 더 있으면 나중에 안 헷갈린다.

src/usage/expected-prices.ts devin-cli/devinswe-1-6 행 - DEVIN_SWE_17을 그대로 재사용한다. swe-1-6과 swe-1-7 list rate가 표에서 정말 같다는 전제다. 다르면 조용한 오추정이 된다.

src/usage/expected-prices.ts gpt-5-6-sol / gpt-6-astra / claude-opus-* / claude-fable-5-1 - OpenAI·Anthropic 공용 상수(GPT56_SOL, GPT6_ASTRA, CLAUDE_OPUS_46, CLAUDE_FABLE_51)를 재사용한다. Devin enterprise list column이 그 숫자와 1원이라도 다르면 Usage 추정이 틀린다. sol은 셀프서브 1.2/6 할인을 의도적으로 버리고 list를 쓴다고 적혀 있으니, docs의 enterprise 열이 정말 4/20/0.4/5인지 스팟체크가 필요하다.

tests/usage/usage-cost.test.ts Devin resolve 스팟체크가 3건(swe-2, swe-1-7-lightning, gpt-5-6-luna)뿐이다. 멤버십 121키 검사는 “행이 있다”만 보고, sonnet-5·kimi·glm·grok 튜플 오타는 못 잡는다. 최소한 sonnet-5와 kimi-k3(또는 kimi-k2-7) 하나씩 더 넣으면 안전하다.

src/providers/registry.ts 정적 seed에 없는 모델(예: 양면 swe-1-6, devinswe-2, devin-cliswe-1-7-lightning)을 overlay에 미리 넣은 것은 liveModels 드리프트 대비로 설명되어 있다. 문제는 아니다. 다만 seed와 overlay 목록이 계속 어긋날 수 있으니, 나중에 live discovery 스냅샷으로 overlay 커버리지를 검사하는 테스트가 있으면 유지보수가 편해진다.

메인테이너의 판단이 필요한 지점

  • Usage UI에 한시 $0/할인 프로모를 절대 넣지 않고 list rate만 쓸지(PR 선택). 프로모 기간에 “실제 청구 0인데 추정은 유료”로 보일 수 있다.
  • gpt-5-6-sol 등 타사 상수 재사용을 허용할지, Devin 표 숫자를 DEVIN_* 상수로 전부 분리할지.
  • devin / devin-cli 행 복제를 유지할지(현재 resolve 계약상 필요), 아니면 provider 별칭 테이블을 둘지(범위 확대).

너의 추천
docs.devin.ai/desktop/models 에서 swe-2, claude-sonnet-5, gpt-5-6-sol(enterprise 열) 세 숫자만 스팟체크하고, bun test tests/usage/usage-cost.test.ts 초록이면 dev에 머지. 프로모 미반영·verified-derived·양면 복제는 현 계약에 맞다. 여력이 있으면 테스트에 sonnet-5·kimi 한 줄씩만 더 넣고 머지해도 된다.

이 댓글은 grok-bot이 작성했습니다

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38c1e245e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// gpt-5-6-sol uses the enterprise list column — the same table's self-serve
// column shows a discounted 1.2/6, and the doc calls the list rate the
// billing rate for overage. glm-5-2 likewise takes the nonzero list column.
{ provider: "devin-cli", modelId: "swe-2", cost4: DEVIN_SWE_2, source: `${DEVIN_SWE2_NOTE}; ${DEVIN_PRICING}`, verifiedAt: "2026-09-13", status: "verified-derived" },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update every structure owner for the pricing change

This changes pricing behavior under src/usage/, but the source-to-doc map assigns that area to both structure/runtime.md and structure/gui-and-management-api.md, and neither is updated; the latter's expected-price-overlay contract currently discusses only the Z.AI derived rows. Update both owning documents in this commit to reflect the new Devin overlays and their subscription/list-rate semantics.

AGENTS.md reference: src/AGENTS.md:L11-L11

Useful? React with 👍 / 👎.

Comment on lines +333 to +336
// Cognition/Devin — the two OAuth surfaces resolve by exact provider id, so
// each carries the roster its liveModels discovery can surface. swe-2 and
// swe-1-6 are listed on both even though each static seed names only one
// side: the live catalog is authoritative and drifts between them.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cover the complete live roster on both Devin provider IDs

The two lists are still based mostly on their static seeds even though fetchDevinUsableModels applies no provider-specific filtering and both registry entries share the union-like DEVIN_MODEL_CONTEXT_WINDOWS. If a devin-cli account discovers gpt-5-6-luna/gpt-5-6-terra, or a devin account discovers gpt-6-astra, the exact overlay lookup misses; the dashed GPT IDs do not match the generated dotted metadata IDs, and Astra has no generated vendor row, so those attempts remain price_unmatched. Register the full discoverable base-model union for both provider namespaces, with tests for the cross-surface cases.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant