Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/adr/0017-simplified-managed-stack-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,17 @@ background task for callers that want full lazy preparation. Eager capability
activation remains independent of this setting.
That runtime session owns the shared preparation fibers.

Lazy traffic capabilities may retire independently after their configured idle timeout (see the
package [traffic stopping documentation](../../packages/stack/README.md)). The traffic policy is
per capability: supported lazy capabilities default to 60 seconds, `false` disables retirement,
eager capabilities never retire, and unsupported capabilities remain opted out. Activity covers
in-flight HTTP requests and streams plus open WebSocket or TCP connections; idle HTTP keep-alive
sockets do not count. The Supervisor serializes workload retirement with activation and lifecycle
operations, and fences new requests for the retiring capability. Unrelated ingress continues;
dependencies stay running while their dependants are active, and Studio with `pg-meta` retires
together. Retained data and listeners let the next request wake the capability and restart its
workloads.

In both modes, lazy activation prepares the requested dependency closure with
bounded concurrency before starting its workloads. Explicit
`stack.prepare(...)` remains available as a cache-only warmup for callers that
Expand Down Expand Up @@ -268,7 +279,7 @@ Tests follow consumed boundaries:
- stack integration covers identity, sticky ports, durable lifecycle,
ownership, stale-owner recovery, and interrupted cleanup;
- supervisor integration covers detached ownership, RPC, stop, and
destroy; and
destroy; dedicated idle-stop integration covers retirement and wake-up; and
- one shared stack-package E2E journey runs in native and Docker modes, starts
with PostgreSQL alone, activates every other service through realistic
traffic, verifies cross-service behavior, then exercises
Expand Down
23 changes: 23 additions & 0 deletions packages/stack/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,29 @@ capability is lazy. Starting the stack therefore launches only
PostgreSQL by default; capabilities configured as eager join its startup dependency closure.
The remaining lazy capabilities activate through the stack's listeners on demand for the current
running session.

Lazy REST, Auth, Realtime, Studio, and pooler capabilities stop after 60 seconds without traffic
by default. Traffic means an active request or stream; idle HTTP keep-alive sockets do not keep a
service running, while open WebSocket or TCP connections do. Configure a different positive
timeout, or disable traffic stopping for a capability, with `idleTimeoutSeconds`:

```ts
await stack.start({
config: {
capabilities: {
rest: { idleTimeoutSeconds: 120 },
realtime: { idleTimeoutSeconds: false },
},
},
});
```

Eager capabilities never auto-stop. PostgreSQL, Storage, Functions, Mail, and Analytics do not opt
into traffic stopping yet. Studio and its `pg-meta` companion are stopped and started together.
Dependency protection keeps required dependencies available while a capability is running.
Stopping preserves listeners and data, and the next request wakes the lazy capability and restarts
its workloads.
Comment thread
jgoux marked this conversation as resolved.

Native workloads have a two-minute readiness budget to allow cold starts to load shared libraries;
container workloads retain a 30-second budget, and PostgreSQL uses its configured `health_timeout`.
Each readiness probe returns immediately when its endpoint becomes healthy.
Expand Down
28 changes: 28 additions & 0 deletions packages/stack/src/gateway/ActivityTracker.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { Effect } from "effect";
import type { CapabilityName } from "../public/Capability.ts";

/** Tracks gateway work that must keep a lazy capability running. */
export interface GatewayActivity {
readonly track: <A, E>(
capability: CapabilityName,
effect: Effect.Effect<A, E>,
) => Effect.Effect<A, E>;
}

export interface GatewayActivityCallbacks {
readonly begin: (capability: CapabilityName) => Effect.Effect<void>;
readonly end: (capability: CapabilityName) => Effect.Effect<void>;
}

/** Adapts gateway lifetimes to the Supervisor-owned traffic controller. */
export const makeGatewayActivity = (
callbacks: GatewayActivityCallbacks,
): Effect.Effect<GatewayActivity> =>
Effect.succeed({
track: (capability, effect) =>
Effect.acquireUseRelease(
callbacks.begin(capability),
() => effect,
() => callbacks.end(capability),
),
} satisfies GatewayActivity);
15 changes: 13 additions & 2 deletions packages/stack/src/gateway/Gateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import type { CapabilityName } from "../public/Capability.ts";
import type { PortField } from "../public/Status.ts";
import { makeHttpGateway, type HttpGateway, type HttpGatewayOptions } from "./HttpGateway.ts";
import { makeTcpGateway, type TcpGateway, type TcpGatewayOptions } from "./TcpGateway.ts";
import type { GatewayActivity } from "./ActivityTracker.ts";

/** A private backend endpoint returned only after activation has completed. */
export interface BackendEndpoint {
Expand Down Expand Up @@ -112,6 +113,8 @@ export interface StackGatewayOptions {
readonly activate: (
capability: CapabilityName,
) => Effect.Effect<ActivationResult, GatewayActivationError>;
/** Optional traffic lease shared by the supervisor's idle-stop controller. */
readonly activity?: GatewayActivity;
}

/** Compose the protocol gateways under one Supervisor-owned lifecycle scope. */
Expand All @@ -135,7 +138,11 @@ export const makeGateway = (
});
}
const acquired = yield* Effect.exit(
makeHttpGateway({ ...entry.options, activate: options.activate }),
makeHttpGateway({
...entry.options,
activate: options.activate,
activity: options.activity,
}),
);
if (Exit.isFailure(acquired)) {
yield* closeValues(http.values());
Expand All @@ -151,7 +158,11 @@ export const makeGateway = (
});
}
const acquired = yield* Effect.exit(
makeTcpGateway({ ...entry.options, activate: options.activate }),
makeTcpGateway({
...entry.options,
activate: options.activate,
activity: options.activity,
}),
);
if (Exit.isFailure(acquired)) {
yield* closeValues([...http.values(), ...tcp.values()]);
Expand Down
14 changes: 9 additions & 5 deletions packages/stack/src/gateway/HttpGateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import type {
PreparedGatewayRoute,
} from "./Gateway.ts";
import { GatewayRouteNotFoundError, isGatewayProxyRoute } from "./Gateway.ts";
import type { GatewayActivity } from "./ActivityTracker.ts";
import type {
HostListener,
HostListenerHttpEvent,
Expand All @@ -48,6 +49,7 @@ export interface HttpGatewayOptions {
) => Effect.Effect<BackendEndpoint, GatewayActivationError>;
readonly cors?: Readonly<Record<string, string>>;
readonly healthPaths?: ReadonlyArray<string>;
readonly activity?: GatewayActivity;
}

export interface HttpGateway {
Expand Down Expand Up @@ -231,9 +233,9 @@ const proxy = (
const onIncomingError = (cause: Error) => {
if (settled) return;
settled = true;
cleanup();
outgoing?.destroy();
incoming?.destroy();
cleanup();
resume(Effect.fail(new GatewayBackendError({ cause })));
};
const onIncomingAborted = () => onIncomingError(new Error("backend response aborted"));
Expand All @@ -250,7 +252,6 @@ const proxy = (
onIncomingError(new Error("response closed"));
};
const cleanup = () => {
if (outgoing !== undefined) outgoing.off("error", onOutgoingError);
request.off("aborted", onRequestAborted);
response.off("close", onResponseClose);
if (incoming !== undefined) {
Expand Down Expand Up @@ -295,9 +296,10 @@ const proxy = (
response.once("close", onResponseClose);
request.pipe(outgoing);
return Effect.sync(() => {
cleanup();
settled = true;
outgoing.destroy();
incoming?.destroy();
cleanup();
});
});

Expand Down Expand Up @@ -381,9 +383,10 @@ const handleRequest = (
proxy(request, response, backend, options, path, headers),
),
);
const tracked = options.activity?.track(route.capability, activation) ?? activation;
// Node invokes this handler outside Effect; use the owner-scoped FiberSet
// runtime so cancellation of the gateway interrupts in-flight activation.
const fiber = runFork(activation);
const fiber = runFork(tracked);
cancelOnRequestClose(request, response, fiber);
fiber.addObserver((exit) => {
if (!Exit.isFailure(exit) || response.writableEnded || response.destroyed) return;
Expand Down Expand Up @@ -515,6 +518,7 @@ const handleUpgrade = (
}),
),
);
const tracked = options.activity?.track(route.capability, activation) ?? activation;
const onSocketClose = () => fiber.interruptUnsafe();
const onSocketEnd = () => fiber.interruptUnsafe();
const onSocketError = () => fiber.interruptUnsafe();
Expand All @@ -523,7 +527,7 @@ const handleUpgrade = (
socket.off("end", onSocketEnd);
request.off("aborted", onRequestAborted);
};
const fiber = runFork(activation);
const fiber = runFork(tracked);
socket.once("close", onSocketClose);
socket.once("end", onSocketEnd);
socket.once("error", onSocketError);
Expand Down
5 changes: 4 additions & 1 deletion packages/stack/src/gateway/TcpGateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import type {
GatewayRouteRequest,
} from "./Gateway.ts";
import type { HostListener } from "../supervisor/HostListener.ts";
import type { GatewayActivity } from "./ActivityTracker.ts";

export interface TcpGatewayOptions {
readonly address?: string;
Expand All @@ -25,6 +26,7 @@ export interface TcpGatewayOptions {
request: GatewayRouteRequest,
activation: ActivationResult,
) => Effect.Effect<BackendEndpoint, GatewayActivationError>;
readonly activity?: GatewayActivity;
}

export interface TcpGateway {
Expand Down Expand Up @@ -140,10 +142,11 @@ const handleConnection = (
return tunnel(source, backend);
}),
);
const tracked = options.activity?.track(route.capability, operation) ?? operation;
source.once("error", onPreActivationError);
// Node invokes this handler outside Effect; use the owner-scoped FiberSet
// runtime for the exact accepted connection's lifecycle.
const fiber = runFork(operation);
const fiber = runFork(tracked);
const onSourceClose = () => fiber.interruptUnsafe();
source.once("close", onSourceClose);
fiber.addObserver(() => {
Expand Down
Loading