NVIDIA · jgarciao · Aug 26, 2026 · Aug 31, 2026
@@ -232,6 +232,128 @@ Secrets and Vault storage backends one at a time:
 mise run e2e:kubernetes:credential-drivers
 ```
 
+### Kubernetes E2E (`e2e/rust/e2e-kubernetes.sh`)
+
+Kubernetes e2e tests deploy an OpenShell gateway into a real Kubernetes cluster
+via Helm and run the Rust e2e suite against it. On vanilla Kubernetes the harness
+reaches the gateway through `kubectl port-forward`; on OpenShift it instead uses a
+passthrough Route secured with mandatory mTLS (see the OpenShift note below).
+
+Run with an ephemeral k3d cluster (macOS; created and torn down automatically):
+
+```shell
+mise run e2e:kubernetes
+```
+
+Target an existing cluster (kind, k3d, or OpenShift):
+
+```shell
+OPENSHELL_E2E_KUBE_CONTEXT=my-context mise run e2e:kubernetes
+```
+
+Scope to a single test for local debugging:
+
+```shell
+OPENSHELL_E2E_KUBE_TEST=smoke mise run e2e:kubernetes
+```
+
+**OpenShift**: when the target cluster exposes the `route.openshift.io` API
+group, the harness automatically applies SCC-compatible Helm overrides, grants
+the required SCCs (`privileged` to `openshell-sandbox`, and `anyuid` to the
+PostgreSQL fixture for DB scenarios), and drives the gateway through a
+passthrough Route with mandatory mTLS instead of port-forward. No extra flags are
+needed, but `oc` must be installed and authenticated against the target cluster
+with permission to modify SCC bindings (`oc adm policy add-scc-to-user`) — the
+harness exits early if `oc` is missing. The SCC grants and extracted client
+mTLS material are removed during cleanup, including on failure or interrupt.
+
+On a **remote** cluster, drop the `e2e-host-gateway` feature. Those tests rely
+on the sandbox-side `host.openshell.internal` alias reaching the machine running
+the tests, which is unreachable from pods on a remote cluster, so they fail.
+Left enabled, the `host_gateway_alias` suite fails because
+`host.openshell.internal` does not resolve inside the pod, so the gateway
+SSRF-denies the request (`DNS resolution failed` / `ssrf_denied`) — a networking
+property of remote pods, not a gateway or transport fault. Override
+`OPENSHELL_E2E_KUBERNETES_FEATURES` to exclude it:
+
+```shell
+OPENSHELL_E2E_KUBE_CONTEXT=$(oc config current-context) \
+  OPENSHELL_E2E_KUBERNETES_FEATURES="e2e,e2e-kubernetes" \
+  mise run e2e:kubernetes
+```
+
+On an existing cluster the harness builds the CLI from your branch but pulls the
+**published** gateway/supervisor image (default tag `latest`). The CLI and the
+image can therefore be different versions. If tests fail because of this version
+difference — for example, sandbox tests fail with `Pod exists with phase: Failed`
+or connect-based tests stall because the deployed image predates a feature your
+branch CLI needs — set `IMAGE_TAG` to an image that matches your branch.
+
+The `latest` tag lags to the last semver release, so it is often older than
+`main`. Two better choices:
+
+- `IMAGE_TAG=dev` — a floating tag that tracks the latest `main` build. Good for
+  an ad-hoc run when your branch is close to `main` HEAD. Because it floats, two
+  runs on different days can pull different images, so it is not reproducible.
+- **Pin the exact commit your branch is based on** — deterministic and immune to
+  a floating tag moving. Published tags are the full 40-char git SHA (semver tags
+  without a `v` prefix also exist but only for released versions):
+
+```shell
+OPENSHELL_E2E_KUBE_CONTEXT=$(oc config current-context) \
+  OPENSHELL_E2E_KUBERNETES_FEATURES="e2e,e2e-kubernetes" \
+  IMAGE_TAG=$(git rev-parse "$(git merge-base HEAD upstream/main)") \
+  mise run e2e:kubernetes
+```
+
+To pin a specific released version, use its semver tag without a `v` prefix
+(`0.0.115`, not `v0.0.115`):
+
+```shell
+OPENSHELL_E2E_KUBE_CONTEXT=$(oc config current-context) \
+  OPENSHELL_E2E_KUBERNETES_FEATURES="e2e,e2e-kubernetes" \
+  IMAGE_TAG=0.0.115 \
+  mise run e2e:kubernetes
+```
+
+A semver tag matches a released commit, which may be behind `main`; if your
+branch CLI needs a newer feature, pin the SHA of your branch's base instead.
+
+Confirm a tag exists before relying on it (set `TAG` to the tag you plan to use):
+
+```shell
+TAG=0.0.115
+skopeo inspect "docker://ghcr.io/nvidia/openshell/gateway:${TAG}"
+```
+
+`IMAGE_TAG` sets only the gateway/supervisor image; the CLI under test is always
+built from your branch. To validate against images from your exact commit
+instead, build and push them and point `OPENSHELL_REGISTRY`/`IMAGE_TAG` at them.
+
+Available task variants:
+
+| Task | Purpose |
+|---|---|
+| `e2e:kubernetes` | Default Rust e2e against Helm-deployed gateway |
+| `e2e:kubernetes:db` | All database backend scenarios (SQLite + external PostgreSQL) |
+| `e2e:kubernetes:sidecar` | Supervisor sidecar topology overlay |
+| `e2e:kubernetes:credential-drivers` | Kubernetes Secrets and Vault credential storage |
+| `e2e:kubernetes:workspace-managed` | Managed workspace mode (auto-created namespaces) |
+| `e2e:kubernetes:workspace-operator` | Operator workspace mode (pre-provisioned namespaces) |
+| `e2e:kubernetes:v1alpha1` | Agent Sandbox v1alpha1 compatibility |
+| `e2e:kubernetes:external-driver` | External Kubernetes driver sidecar |
+
+Kubernetes e2e environment variables:
+
+| Variable | Purpose |
+|---|---|
+| `OPENSHELL_E2E_KUBE_CONTEXT` | kubectl context for an existing cluster (skips k3d creation) |
+| `OPENSHELL_E2E_KUBE_TEST` | Scope to a single test (e.g. `smoke`) |
+| `OPENSHELL_E2E_KUBE_EXTRA_VALUES` | Colon-separated additional Helm values files |
+| `OPENSHELL_E2E_KUBERNETES_FEATURES` | Cargo feature flags (default: `e2e,e2e-host-gateway,e2e-kubernetes`) |
+| `IMAGE_TAG` | Gateway/supervisor image tag (default: `latest` for existing clusters) |
+| `OPENSHELL_REGISTRY` | Image registry prefix (default: `ghcr.io/nvidia/openshell`) |
+
 Run a single test directly with cargo:
 
 ```shell
@@ -263,3 +385,5 @@ The harness (`e2e/rust/src/harness/`) provides:
 | `OPENSHELL_GATEWAY_ENDPOINT` | Run E2E tests against an existing plaintext HTTP gateway endpoint |
 | `OPENSHELL_E2E_DRIVER` | Driver name exported by the e2e gateway wrapper (`docker`, `podman`, or `vm`) |
 | `OPENSHELL_E2E_CREDENTIAL_DRIVERS` | Enables the Kubernetes credential-driver fixture path in `e2e/with-kube-gateway.sh` |
+| `OPENSHELL_E2E_KUBE_CONTEXT` | kubectl context for Kubernetes e2e (skips ephemeral k3d) |
+| `OPENSHELL_E2E_KUBE_TEST` | Scope Kubernetes e2e to a single test by name |
@@ -0,0 +1,53 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# OpenShift overlay for the Kubernetes e2e harness.
+#
+# Bundles the OpenShift-specific settings the harness needs: the Route/mTLS
+# transport, and an image pull policy that avoids stale cached images.
+#
+# Route/mTLS transport: on OpenShift, `kubectl port-forward` stalls the SSH-relay
+# `sandbox connect` path (round-trip-heavy SSH over SPDY), so the harness drives
+# the gateway through a passthrough OpenShift Route with mTLS instead. This
+# overlay turns TLS back on (values-skaffold.yaml disables it), enables the Route,
+# and promotes the cert-verified caller to a dev principal.
+#
+# Image pull policy: force `Always` so runs against the `latest` upstream image
+# actually use it, instead of a stale copy cached on the cluster nodes.
+#
+# Layered by e2e/with-kube-gateway.sh AFTER ci/values-skaffold.yaml and
+# ci/values-openshift-scc.yaml when an OpenShift cluster is detected. The harness
+# supplies `openshiftRoute.host` and `pkiInitJob.serverDnsNames[0]` via --set at
+# install time (both are the cluster-derived Route hostname).
+#
+# Security: this is NOT an open gateway. `server.tls.clientCaSecretName` defaults
+# to `openshell-server-client-ca` and there is no OIDC, so `require_client_auth`
+# is true and mTLS is MANDATORY at the TLS handshake — a caller with only the
+# Route URL and no client certificate is rejected before any RPC. The passthrough
+# Route terminates TLS at the gateway pod, so this holds end-to-end.
+# `allowUnauthenticatedUsers` only promotes the already cert-verified caller to a
+# dev principal at the app layer (mtls_auth is unsupported with the Kubernetes
+# driver). Both are required together; the client certificate is the access gate.
+image:
+  pullPolicy: Always
+
+supervisor:
+  image:
+    pullPolicy: Always
+
+server:
+  disableTls: false
+  auth:
+    allowUnauthenticatedUsers: true
+
+openshiftRoute:
+  enabled: true
+  # host is supplied via --set at install time (cluster-derived Route hostname).
+  # The default HAProxy Route timeout is 30s, which severs long-lived transfers
+  # (large sandbox upload/download, SSH-relay `sandbox connect`) mid-stream. Raise
+  # both the connection timeout and the passthrough tunnel timeout so these paths
+  # survive. Passthrough Routes proxy in TCP mode, so timeout-tunnel governs the
+  # established tunnel while timeout covers the pre-tunnel phase.
+  annotations:
+    haproxy.router.openshift.io/timeout: 300s
+    haproxy.router.openshift.io/timeout-tunnel: 300s
@@ -0,0 +1,20 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# OpenShift SCC compatibility overlay. Removes the hardcoded runAsUser and
+# fsGroup so that OpenShift's restricted-v2 SCC can inject the namespace-
+# assigned UID/GID range. Layer after values.yaml:
+#   helm install openshell deploy/helm/openshell -f ci/values-openshift-scc.yaml
+#
+# The e2e Kubernetes harness applies this automatically when it detects an
+# OpenShift cluster (route.openshift.io API present).
+
+podSecurityContext: null
+
+securityContext:
+  runAsNonRoot: true
+  runAsUser: null
+  allowPrivilegeEscalation: false
+  capabilities:
+    drop:
+      - ALL
@@ -34,8 +34,10 @@ spec:
         - host.docker.internal
         - host.openshell.internal
   {{- end }}
+  {{- with .Values.podSecurityContext }}
   securityContext:
-    {{- toYaml .Values.podSecurityContext | nindent 4 }}
+    {{- toYaml . | nindent 4 }}
+  {{- end }}
   containers:
     - name: openshell-gateway
       securityContext: