Skip to content

fix: Send request queue write fields under the names the API declares - #1026

Draft
vdusek wants to merge 8 commits into
masterfrom
fix/request-draft-drops-user-data
Draft

fix: Send request queue write fields under the names the API declares#1026
vdusek wants to merge 8 commits into
masterfrom
fix/request-draft-drops-user-data

Conversation

@vdusek

@vdusek vdusek commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

add_request and batch_add_requests sent request fields snake_cased. The API declares its write bodies with additionalProperties: false, so it rejected the whole write with HTTP 400 — a request carrying user_data, no_retry, retry_count, loaded_url, error_messages, or handled_at could not be added at all. The identical dict passed to update_request worked:

add_request     -> 400  {"uniqueKey": "k", "url": "...", "user_data": {...}, "no_retry": true}
update_request  -> 200  {"uniqueKey": "k", "url": "...", "userData":  {...}, "noRetry":  true}

RequestDraft declared only id/unique_key/url/method and left the rest to extra='allow', which alias_generator=to_camel never touches. The spec uses it only for unprocessedRequests in responses and declares both add-request bodies as RequestBase — the client was using a response schema as its input model. The codegen postprocessor now reparents RequestDraft onto RequestBase. Both upstream spec fixes (#952, apify-docs#2774) are closed.

Three things to know:

  • Validation narrows. The newly declared fields are validated instead of passed through, so retry_count='abc' or a naive handled_at now raise ValidationError instead of a 400 from the API.
  • update_request's timestamp format changes (fa77d88): mode='json' emits ISO 8601 where python mode emitted 2019-06-16 10:23:31.607000+00:00. The API accepts both, so this is spec fidelity, not a fix — its own commit, revertible alone.
  • RequestDraftDict loses the static Required marker on unique_key/url — PEP 589 forbids redeclaring a base's key. Pydantic still enforces both at runtime.

Integration tests round-trip every field through both add paths against the live API; they fail on master with InvalidRequestError.

✍️ Drafted by Claude Code

`RequestDraft` declared only id/unique_key/url/method and leaned on
`extra='allow'` for the rest, but `alias_generator=to_camel` never touches
extras. So `user_data`, `no_retry`, `headers`, `payload`, `retry_count`, and
`handled_at` reached the API snake_cased on `add_request` and
`batch_add_requests`, which the API ignores silently. `update_request` was
unaffected - it uses `Request`, which declares the full shape.

The spec declares those request bodies as `RequestBase` and uses
`RequestDraft` only for `unprocessedRequests` in responses, so the codegen
postprocessor now reparents `RequestDraft` onto `RequestBase`.

`RequestDraftDict.unique_key` and `.url` are no longer statically required:
PEP 589 forbids a TypedDict subclass from redeclaring a key of its base.
Pydantic still enforces both at runtime, before any HTTP call.
@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Aug 21, 2026
@vdusek vdusek self-assigned this Aug 21, 2026
@github-actions github-actions Bot added this to the 147th sprint - Tooling team milestone Aug 21, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 21, 2026
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.17%. Comparing base (df80da1) to head (ada2306).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1026      +/-   ##
==========================================
+ Coverage   94.95%   98.17%   +3.21%     
==========================================
  Files          58       58              
  Lines        5436     5426      -10     
==========================================
+ Hits         5162     5327     +165     
+ Misses        274       99     -175     
Flag Coverage Δ
integration 91.92% <33.33%> (+0.20%) ⬆️
unit 89.64% <100.00%> (+3.82%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek changed the title fix: Stop dropping user_data and other fields when adding queue requests fix: Send request queue write fields under the names the API declares Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants