Skip to content

fix: Stop auth refresh race from logging out - #174

Merged
mattdjenkinson merged 2 commits into
mainfrom
fix/auth-refresh-race
Aug 22, 2026
Merged

fix: Stop auth refresh race from logging out#174
mattdjenkinson merged 2 commits into
mainfrom
fix/auth-refresh-race

Conversation

@mattdjenkinson

Copy link
Copy Markdown
Contributor

Summary

Users reported tunnels going dead overnight while Datum Desktop still looked logged in, or flipping to logged-out state after a refresh failure. The underlying issue was concurrent OIDC refresh calls reusing the same refresh token after Zitadel had already rotated it. The loser got RefreshTokenInvalid, the client treated that as a real logout, heartbeats stopped, and every hostname on the connector started returning 503.

This change serializes refresh behind a mutex so only one exchange runs at a time, reloads tokens after acquiring the lock, and skips redundant refreshes when a waiter already picked up a fresh token. When Zitadel rejects a refresh token as invalid or already rotated, we keep the current session instead of clearing auth, matching how cloud-portal handles the same error. Genuine failures (network errors, missing refresh token with no valid access token path) still log the user out. Successful refreshes now log at info with expires_at so default ui.log shows granted refreshes, not just warnings.

Related analysis: datum-cloud/app#48

Test plan

  • cargo test -p lib (46 tests, including 12 refresh-race tests)
  • Concurrent refresh test confirms max one active refresh and coalesced waiters
  • Rotation race keeps session with valid and just-expired access tokens
  • Non-revoked refresh failure still clears auth state
  • force_refresh and load_refreshed paths covered by unit tests
  • Staging build soak test (tunnel stays up overnight without logout)
  • Manual check: ui.log shows Access token refreshed on timer refresh

Related to #48

Concurrent OIDC refresh calls could reuse a rotated refresh token.
Zitadel returned RefreshTokenInvalid and the client cleared auth,
which stopped heartbeats and dropped published tunnels.

Serialize refreshes behind a mutex, keep the session on rotation races
(the same rule cloud-portal uses), and add unit tests for the paths
that caused daily logouts in #48.
@mattdjenkinson
mattdjenkinson marked this pull request as ready for review August 21, 2026 09:11
@mattdjenkinson mattdjenkinson self-assigned this Aug 21, 2026
@mattdjenkinson
mattdjenkinson merged commit b1028f5 into main Aug 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants