Skip to content

fix(agent): harden PSU Event Hub worker lifecycle#1830

Open
Marc-André Moreau (mamoreau-devolutions) wants to merge 1 commit into
masterfrom
mamoreau-devolutions-psu-agent-review-plan
Open

fix(agent): harden PSU Event Hub worker lifecycle#1830
Marc-André Moreau (mamoreau-devolutions) wants to merge 1 commit into
masterfrom
mamoreau-devolutions-psu-agent-review-plan

Conversation

@mamoreau-devolutions

Copy link
Copy Markdown
Contributor

Hardens the experimental PowerShell Universal Event Hub compatibility path so agent deployments are less likely to exhaust local PowerShell workers, leak completed results, or keep stuck scripts running indefinitely.

The agent now uses safer defaults for worker execution, reconnects more gradually after hub failures, and rejects unsupported default-credential configuration instead of attempting unauthenticated connections.

Issue: #1803

Share the PowerShell worker pool, bound stored results, enforce worker timeouts, and make SignalR reconnects and execution tasks shut down predictably.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mamoreau-devolutions

Copy link
Copy Markdown
Contributor Author

Implementation notes:

  • Shares a single PSU PowerShell worker across Event Hub connections and AppToken secret resolution, so MaxWorkerPoolSize applies at the task level instead of per connection.
  • Keeps execution tasks alive across SignalR reconnects and shuts them down only when the connection task is stopping, preserving returned execution IDs through transient disconnects.
  • Adds a 30-minute PowerShell execution timeout, kill-on-drop child handling, tempfile-backed request files, and a stable worker script file per worker lifetime.
  • Bounds stored results with a 15-minute TTL and 1024-entry per-connection cap to avoid unbounded memory growth when hubs abandon result IDs.
  • Adds capped exponential reconnect backoff, sanitizes PSU description headers, uses Windows SAM-compatible identity lookup for service-context headers, and skips unsupported UseDefaultCredentials-only connections.

Testing:

  • cargo +nightly fmt --all
  • cargo test -p devolutions-agent -- --test-threads=1
  • cargo clippy -p devolutions-agent --tests -- -D warnings
  • cargo +nightly clippy -p devolutions-agent --tests --no-deps -- -D warnings

Full workspace validation notes:

  • cargo +nightly clippy --workspace --tests -- -D warnings is currently blocked by unrelated pre-existing clippy failures in other crates.
  • cargo test --workspace reaches and passes devolutions-agent tests, then is blocked by devolutions-agent-updater requiring elevation on this machine.

@github-actions

Copy link
Copy Markdown

Let maintainers know that an action is required on their side

  • Add the label release-required Please cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module) when you request a maintainer to cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module)

  • Add the label release-blocker Follow-up is required before cutting a new release if a follow-up is required before cutting a new release

  • Add the label publish-required Please publish libraries (`Devolutions.Gateway.Utils`, OpenAPI clients, etc) when you request a maintainer to publish libraries (Devolutions.Gateway.Utils, OpenAPI clients, etc.)

  • Add the label publish-blocker Follow-up is required before publishing libraries if a follow-up is required before publishing libraries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant