fix: prevent StreamedLog stop() from hanging on a silent stream - #825
Merged
Merged
Conversation
vdusek
added
adhoc
github-actions
Bot
added
the
tested
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #825 +/- ##
===========================================
+ Coverage 83.46% 94.68% +11.22%
===========================================
Files 48 48
Lines 5043 5044 +1
===========================================
+ Hits 4209 4776 +567
+ Misses 834 268 -566
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
janbuchar
approved these changes
May 25, 2026
vdusek
added a commit
that referenced
this pull request
Sep 16, 2026
…or call (#1027) ### Description `actor.call()` with the default `logger='default'` could raise on a run that had already succeeded. A status poll that failed after retries left the exception on the watcher task; `stop()` then cancelled an already-finished task and `await`ed it, re-raising past `except asyncio.CancelledError`. The sync twin leaked the same failure out of its thread. - Both watcher loops now use the handler `StreamedLog._stream_log` already had: warn on `is_timeout_error`, `logger.exception` otherwise. A failed poll gets reported and the run's own result stands. - The sync watcher's thread is a daemon, so a watcher whose loop never terminates cannot hold up interpreter shutdown. - `StreamedLog.stop()` closes the stream response and caps its join at `_stop_timeout_s` (5s), where it used to join unbounded on a stream that may not speak for hours. - `start()` on both sync classes checks `is_alive()`, so a handle left by a thread that outlived its stop does not block a restart. The close alone is not enough. On Impit 0.13.2 `Response.close()` returns immediately and sets `is_closed=True`, yet a reader blocked in `iter_bytes()` stays blocked, and closing the `Client` does not release it either. The close is kept because it is correct for a custom HTTP client, and the capped join is what makes `stop()` return. ### Why `stop()` is bounded this way - #825 bounded `stop()` by requesting the log stream with a 30s read timeout. - #944 removed that: Impit applies `timeout` to the whole request, streamed body included, so a run logging past the bound was cut off mid-stream with `impit.TimeoutException` (#945). The stream has used `no_timeout` since, with the unbounded `stop()` recorded as a known limitation. - This PR caps the join and closes the response, leaving the stream request alone. `_stream_timeout` stays `no_timeout`, still asserted by `test_streamed_log_sync_requests_stream_with_no_timeout`, so #945 cannot regress. What it fixes is the case #944 left open: a manual `stop()` on a stream gone quiet. Trade-off: a thread outliving the 5s cap keeps running as a daemon, so `stop()` can return before the tail is flushed. On the `actor.call()` path the run's EOF ends the thread first, measured at 0.000s over repeated `apify/hello-world` runs. ### Test plan - 8 unit tests and an integration test that fails a real run's background status polls, each red before the fix. *✍️ Drafted by Claude Code*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In
StreamedLog(src/apify_client/_streamed_log.py),stop()sets_stop_logging = Trueand thenjoin()s the streaming thread. The thread is parked insideiter_bytes()on a blocking socket read, so the flag isn't observed until the next chunk arrives or the long-polling server-side timeout (~360s) elapses —stop()can block for minutes on a quiet actor.Passing a
_read_timeout(30s) to_log_client.stream()caps how longiter_bytes()can block, sostop()unblocks within that window. The timeout is exposed as aClassVar[timedelta]so it can be tuned (or shortened in tests) without monkeypatching a module global. The loop is also wrapped intry/finallyso the buffered tail is flushed even if a read times out.Tradeoff: the sync stream now ends if the actor goes silent for more than 30s (vs. the prior 360s). This is documented on the class attribute and is the intended cost of bounded
stop().A regression test covers the bug.