feat(renderers): add Inkling and Inkling-Small - #103
Merged
Conversation
ApprovabilityVerdict: Needs human review This PR introduces a substantial new renderer (~1000 lines of code) supporting Inkling/Inkling-Small models with multimodal capabilities (image and audio), new configuration options, and new parsing logic. The scope and complexity of this new feature warrants human review. No code changes detected at You can customize Macroscope's approvability policy. Learn more. |
hallerite
force-pushed
the
feat/inkling-renderer
branch
from
August 11, 2026 21:35
f235f2a to
7a17ef0
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7b2a55b. Configure here.
snimu
approved these changes
Aug 12, 2026
Implement InklingRenderer for thinkingmachines/Inkling — a token-delimited
vision+audio model whose chat template is a flat stream of role-tagged,
<|end_message|>-terminated blocks (no BOS) with a one-off "Thinking effort
level: {N}" system line and <|content_model_end_sampling|> as the assistant
close/eos.
- Text / tools / reasoning / reasoning_effort: byte-parity with
apply_chat_template across all shapes (effort labels + raw floats, tool
cycles, multi-tool-call, content=None, tool-name resolution via
tool_call_id).
- Image + audio: byte-parity with the native InklingProcessor, including the
Qwen-VL-style placeholder expansion (image -> num_patches image pads, audio
-> one audio pad per mel frame); ships pixel_values / audio_input_ids in
MultiModalData with exact placeholder anchoring.
- Full parse_response (native-JSON tool args, types preserved) and
bridge_to_next_turn (mm carry-forward; refuses tool messages needing
cross-turn name resolution).
The InklingProcessor is native in transformers >= 5.14 (merged upstream
2026-07-15), so it loads without trust_remote_code. Bump the transformers
floor to >=5.14.0 and fast-track it past the 7-day uv exclude-newer cooldown
(drop the fast-track once 5.14.x clears the window). The full existing suite
was re-validated on 5.14.1 with no regressions.
Wiring: InklingRendererConfig (reasoning_effort str|float, validated),
MODEL_RENDERER_MAP, MULTIMODAL_MODELS={"image","audio"}, registry, __init__
exports. Tests: tests/test_inkling.py plus Inkling rows in the shared
render/config-parity/multimodal matrices (extended with an "inkling" family
and an audio modality kit). Docs updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Condense the verbose rationale comments added with the Inkling renderer (MODEL_RENDERER_MAP / MULTIMODAL_MODELS entries, INKLING_EFFORT_MAP, the transformers pin, and the shared-test rows) to terse one-liners matching the surrounding style. Comments only — no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- bridge_to_next_turn copied only the outer mm_data dicts, so appending a carried-forward modality's items mutated the caller's previous_multi_modal_data lists in place. Copy the per-modality lists too. Adds a regression test (test_inkling_bridge_does_not_mutate_prior_mm_data). - _audio_hash now folds in the sampling rate, so mm_hashes["audio"] uniquely identifies the clip (matching the processor cache key); identical samples at different rates no longer collide as media IDs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hallerite
force-pushed
the
feat/inkling-renderer
branch
from
August 12, 2026 18:46
e22d00f to
de48898
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Adds one
InklingRenderershared by both Thinking Machines checkpoints:thinkingmachines/Inklingthinkingmachines/Inkling-SmallThe two pinned checkpoints ship byte-identical chat templates, tokenizer assets, and processor configuration, so they intentionally use the same renderer and config. Each model ID is independently covered by the shared render, config, and multimodal parity matrices.
This PR is stacked on #123. Merge the Gemma 4 PR first, then this PR.
What it does
reasoning_effortrender with strict chat-template parity.InklingProcessor, including dynamic placeholder expansion andMultiModalDataanchoring.parse_responsepreserves native JSON tool arguments and separates reasoning/content channels.bridge_to_next_turncarries multimodal state without mutating prior data.Dependency
Keeps the repository-wide
transformers>=4.50.0floor so renderers remains co-installable with prime-rl's current Transformers pin. Inkling tokenizer and text-only rendering stay usable on older releases. Image/audio processing fails lazily with a clear upgrade message when nativeInklingProcessorsupport is unavailable (introduced in Transformers 5.14). The lock continues to resolve totransformers==5.14.1.Review follow-up
audio=callaudio_cache_maxboundVerification
git diff --checkpasshallerite <git@hallerite.com>using the configured SSH signing keyNote
Medium Risk
Large new renderer (~1k lines) on the critical render/parse/bridge path; image/audio depend on Transformers 5.14 at runtime though the declared floor stays lower.
Overview
Adds
InklingRendererforthinkingmachines/InklingandInkling-Small, sharing one implementation because both checkpoints use identical template/tokenizer/processor assets.The renderer matches
apply_chat_templatefor text, tools,reasoning_effort(injected asThinking effort level: {N}), and assistant reasoning blocks; image/audio paths expand placeholders like nativeInklingProcessor(Transformers ≥5.14) and attachMultiModalData.parse_inklingdecodes completions into content, reasoning, and native-JSON tool calls.bridge_to_next_turnuses thinking retentionall, merges multimodal sidecars without mutating prior data, and refuses extensions that need unresolved tool names.InklingRendererConfigaddsreasoning_effort,image_cache_max, andaudio_cache_max. The package keepstransformers>=4.50.0for co-install with prime-rl; multimodal fails lazily with an upgrade hint whenInklingProcessoris missing. Docs, registry, and shared parity/bridge/multimodal tests are wired for both model IDs;uv.lockbumps totransformers==5.14.1.Reviewed by Cursor Bugbot for commit de48898. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Add Inkling and Inkling-Small renderers with image/audio multimodal support
InklingRendererin renderers/inkling.py with full rendering, parsing, bridging, and multimodal (image + audio) support forthinkingmachines/Inklingandthinkingmachines/Inkling-Small.Gemma4Rendererin renderers/gemma4.py with image support and model-specific empty-thought prefill handling forgoogle/gemma-4-*checkpoints.InklingRendererConfig(withreasoning_effort,image_cache_max,audio_cache_max) andGemma4RendererConfig(withenable_thinking,preserve_thinking,image_cache_max) to renderers/configs.py.parse_inklingto renderers/parsing.py for decoding Inkling model outputs into structuredParsedResponsewith tool call extraction._build_gemma4_featuresto serialize Gemma 4 image inputs into vLLM-compatible kwargs; raisesRuntimeErrorif torch or required vLLM modules are absent.transformers>=5.14; text-only rendering works on older releases.Macroscope summarized e22d00f.