Skip to content

Unify Docker image by removing SageMaker variant - #1773

Open
kmcginnes wants to merge 51 commits into
mainfrom
remove-sagemaker-docker-image
Open

kmcginnes wants to merge 51 commits into
mainfrom
remove-sagemaker-docker-image

Conversation

@kmcginnes

@kmcginnes kmcginnes commented May 20, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Graph Explorer now finds its proxy server on its own. Users no longer configure a "Public or Proxy Endpoint" or tick a "Using Proxy-Server" checkbox, so the connection form is just the database URL plus optional IAM settings. Every connection goes through the proxy server by default.

This also removes the separate SageMaker Docker build. One image serves every deployment mode, standalone, SageMaker, and arbitrary reverse proxies, with no build-time path configuration.

Connecting straight from the browser to the database is still possible, but it's now a deprecated option under Advanced options and will be removed in a follow-up PR once we've heard from anyone who depends on it.

Key changes:

  • The client derives API endpoints at runtime by cutting the /explorer static-mount segment out of location.pathname, so it works behind any reverse proxy at any prefix. A reverse proxy that renames the segment away gets a clear "Reverse proxy misconfigured" error instead of requests going to the wrong place
  • url (the proxy endpoint) is gone from the connection model. graphDbUrl is the single endpoint field, and proxyConnection: false marks a deprecated direct connection
  • The "Graph Connection URL" field is now "Database URL"
  • A direct connection sends no proxy or IAM headers and needs an absolute http(s) URL. When the browser can't reach its database, it shows "Database not reachable from the browser" with the fix, and an http:// database from an https:// page gets its own "Insecure database URL" error
  • Direct connections are marked "Direct" in the connection list and "Direct from browser (deprecated)" in the connection details, with a tooltip explaining how to switch
  • NEPTUNE_NOTEBOOK=true becomes a runtime-only preset for port 9250, cloudwatch logs, and SSL off
  • CI builds one image and publishes it under both the regular and sagemaker-* tags during the transition
  • Legacy stored connections are transformed at the storage boundary, so every reader sees the current shape. Existing direct connections stay direct
  • Legacy PUBLIC_OR_PROXY_ENDPOINT and USING_PROXY_SERVER environment variables are still honored, so an existing deployment needs no change
  • The connection details panel clamps a long database URL to two lines instead of scrolling sideways

Breaking changes

The client now finds the proxy at the parent of its own /explorer path. So the UI has to be served by the proxy server, directly or through a same-origin reverse proxy that forwards that path, and split-origin UI hosting, where a connection pointed at a proxy somewhere else, no longer works. PROXY_SERVER_CORS_ORIGIN stays, because it still governs other web origins calling the proxy API.

A reverse proxy that renames the /explorer segment away, for example mapping /gx/ onto /explorer/, now shows "Reverse proxy misconfigured". Forward the segment intact, at any prefix.

With one image, setting NEPTUNE_NOTEBOOK=true with -e or in config.json now applies the notebook preset's port 9250 and cloudwatch logs. The standard image used to keep port 80 there. Set PROXY_SERVER_HTTP_PORT if you want a different port. The SageMaker lifecycle script already passes NEPTUNE_NOTEBOOK=true and expects 9250, so notebooks are unaffected.

The sagemaker-* tags are now aliases of the standard image, so they no longer bake in the notebook defaults. Running one without NEPTUNE_NOTEBOOK=true serves HTTPS on 443 like the standard image. Pass NEPTUNE_NOTEBOOK=true to keep port 9250 and cloudwatch logs.

A mounted .env that sets PROXY_SERVER_HTTP_PORT or LOG_STYLE now takes effect. The standard image used to override both with port 80 and default logs.

A stored direct connection whose URL isn't an absolute http(s) URL, such as a relative /neptune, now shows an "Invalid URL" error until the connection is edited.

Downgrading to an older image after editing or exporting connections isn't supported, because older builds can't read connections without the proxy url field.

Deprecated

Direct browser-to-database connections. They need the database to allow cross-origin requests, and they miss the proxy's IAM signing, server-side logging, and PROXY_SERVER_ALLOWED_DB_ORIGINS enforcement. The follow-up that removes them will carry its own migration notes.

Screenshots

Validation

  • 3012 tests passing across 230 files
  • pnpm checks clean: types, lint, format
  • Exercised by hand against a live Neptune 1.4.7.0 air-routes cluster with IAM SigV4, in three topologies: the express server at /explorer/, a reverse proxy at /proxy/9250/explorer/ that strips the prefix, and the Vite dev server
  • Tested on a SageMaker Neptune notebook with no lifecycle script changes, before the direct option was added back. The proxied request code was refactored since, with the same behavior, covered by unit tests
  • Exercised the direct option in a browser against Gremlin Server, with and without CORS: form-created, imported from a legacy file, and from USING_PROXY_SERVER=false
  • Manual testing:
    • Deployed behind a SageMaker reverse proxy at /proxy/9250/explorer/, relative URLs resolve correctly
    • Existing lifecycle script passes legacy env vars, which the server now resolves rather than ignoring
    • Created and edited connections in the UI, form shows only Database URL plus IAM fields, with the direct option under Advanced options
    • Imported legacy connection files covering all four url/proxyConnection combinations, migration works and direct connections stay direct
    • A direct connection sends no proxy or IAM headers, and turning the option off moves it to the proxy

Related Issues

#1623 is delivered with a documented difference from its written criteria: the logger resolves through apiUrl() rather than an apiBase constant. #1624's read-time transform ships here instead of a startup migration, and the follow-up finishes it by dropping proxyConnection. Each issue carries a comment explaining why.

#1772 listed renaming the "Graph Connection URL" label as out of scope. I renamed it to "Database URL" anyway, since it's now the only endpoint field on the form.

The /api route-prefix work is not part of this PR and stays deferred under #2016.

Check List

  • I confirm that my contribution is made under the terms of the Apache 2.0 license.
  • I have verified pnpm checks passes with no errors.
  • I have verified pnpm test passes with no failures.
  • I have covered new added functionality with unit tests if necessary.
  • I have updated documentation if necessary.

@kmcginnes
kmcginnes force-pushed the remove-sagemaker-docker-image branch 3 times, most recently from a109b19 to 1a8dad0 Compare June 18, 2026 22:23
@kmcginnes kmcginnes changed the title feat: unify Docker image by removing SageMaker variant Unify Docker image by removing SageMaker variant Jun 18, 2026
--env IAM=${IAM} \
--env AWS_REGION=${AWS_REGION} \
--env SERVICE_TYPE=${SERVICE} \
--env PROXY_SERVER_HTTPS_CONNECTION=false \

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be kept

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 1e33ca0.

--env IAM=${IAM} \
--env AWS_REGION=${AWS_REGION} \
--env SERVICE_TYPE=${SERVICE} \
--env PROXY_SERVER_HTTPS_CONNECTION=false \

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be kept

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 1e33ca0.

Comment thread docs/references/configuration.md Outdated
Comment on lines +108 to +115
### `NEPTUNE_NOTEBOOK`

Runtime convenience preset for SageMaker/Jupyter deployments. When set to `true`, configures port 9250, cloudwatch logging, and disables SSL automatically.

- Optional
- Default: not set
- Type: `boolean`

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to document this

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 0095edb.

@kmcginnes
kmcginnes force-pushed the remove-sagemaker-docker-image branch from 1a8dad0 to 772cdf0 Compare June 30, 2026 23:52
@kmcginnes
kmcginnes force-pushed the remove-sagemaker-docker-image branch from 772cdf0 to 287ecf5 Compare September 9, 2026 19:41
@mjuarros

Copy link
Copy Markdown
Contributor

Pre-existing follow-up findings

These are not introduced by this PR and should not block it, but they surfaced while tracing the updated proxy path and are worth validating separately:

  • When PROXY_SERVER_ALLOWED_DB_ORIGINS is unset, the proxy accepts any HTTP(S) destination supplied through graph-db-connection-url. An exposed deployment can therefore act as a server-side request proxy to internal services. Consider a follow-up issue to evaluate a deny-by-default or administrator-managed allowlist.
  • Error handling logs graph-db-connection-url verbatim. URLs containing user information could expose credentials in logs. Consider redacting the value to scheme, hostname, and port.
  • process-environment.sh interpolates configuration values directly into .env and JSON. Quotes, backslashes, commas, or newlines can corrupt the generated files. Consider replacing the shell serialization with the existing Node runtime or another real JSON serializer.
  • A current pnpm audit --audit-level high reports dependency updates needed for browserslist and the pinned pnpm version. This is independent of the PR implementation and can be tracked as dependency maintenance.

If these behaviors are still considered valid after maintainer review, I recommend tracking each concern in a separate issue so this PR can stay focused.

"My Connection",
);
await user.type(
screen.getByRole("textbox", { name: "Graph Connection URL" }),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we expand this suite to cover the Connection form behavior required by #1772? In addition to the URL validation already covered, please verify that the Public or Proxy Endpoint field and Using Proxy-Server checkbox are absent, the Graph Connection URL and IAM controls are available without another toggle, and create/edit submissions contain the canonical graphDbUrl shape without legacy url or proxyConnection fields. These assertions protect the main user-facing simplification and backward-compatible edit path from regression.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in ffe0b3e. The suite now checks that the proxy endpoint field and the proxy checkbox are gone, that IAM auth shows up without another toggle, and that the saved connection has only graphDbUrl with no url or proxyConnection.

Create and edit both go through mapToConnection, which builds the connection from an explicit field list. Legacy fields are stripped when the stored configuration is read (df95624), so edit can't carry them forward either.

Comment thread process-environment.sh Outdated
GRAPH_EXP_HTTPS_CONNECTION="false"
if [ "$NEPTUNE_NOTEBOOK" = "true" ]; then
# Force SSL off for Neptune Notebook environments
PROXY_SERVER_HTTPS_CONNECTION="false"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not reliably force proxy HTTPS off when the container already has PROXY_SERVER_HTTPS_CONNECTION=true. The script writes false to .env, but node-server.ts loads it with the default dotenv.config() behavior, which does not override the inherited process environment. The entrypoint can therefore skip certificate generation after reading false from .env, while Node still reads the inherited true and attempts to start HTTPS without certificates. Please make the forced notebook value effective in the Node process and add a test with conflicting HTTPS environment variables that asserts the parsed runtime configuration has SSL disabled.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I handled this in #2252 instead of forcing the value inside Node. When NEPTUNE_NOTEBOOK and PROXY_SERVER_HTTPS_CONNECTION are both true, the server now fails at env parsing with an error that names both variables and says how to fix it, so the certificate path never runs. Tests with the conflicting values are in env.test.ts. The lifecycle script still sets PROXY_SERVER_HTTPS_CONNECTION=false, so the notebook path is unaffected.

@@ -124,19 +122,16 @@ To provide a default connection such that initial loads of Graph Explorer always
These are the valid environment variables used for the default connection, their defaults, and their descriptions.

- Required:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also remove the note above that says direct database connections can bypass PROXY_SERVER_ALLOWED_DB_ORIGINS? This PR removes direct mode, so the note now describes an impossible configuration. The documentation should state that the allowlist applies to every database request because all requests route through the proxy.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in f7ea1a0. The note now says the allowlist applies to every database request because the client always goes through the Proxy Server (wording tightened in 4038b93). Same change in security.md.

Comment thread CONTEXT.md Outdated
_Avoid_: Save-status indicator

**Proxy Server**:
The Node.js server that serves the frontend (mounted at `/explorer`) and proxies all database requests (mounted at `/`). The client resolves API endpoints relative to its own origin via `apiUrl()` — `../endpoint` from the static mount — so the frontend and proxy are always same-origin. This means every database request routes through the Proxy Server, which has network access to the database and handles SigV4 signing. See ADR `unify-docker-image-remove-sagemaker-variant`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please carry this proxy-only invariant into docs/agents/product.md as well. Its persistence section still says graph data is queried directly from connected databases, which is no longer accurate. It should clarify that data is queried live through the stateless Graph Explorer proxy and is not persisted by Graph Explorer.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2a7feda. product.md now says graph data is queried live through the Proxy Server and isn't owned or persisted by Graph Explorer.

// connections stored it in `url`. The final `connection.graphDbUrl` fallback
// covers already-migrated data where `url` is absent, and the empty-string
// fallback keeps the result valid when no URL is present at all.
const graphDbUrl = proxyConnection ? connection.graphDbUrl : url;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the previous fallback semantics when proxyConnection is absent. Before this PR, normalizeConnection inferred proxy mode whenever graphDbUrl existed. A legacy connection containing both url (the proxy endpoint) and graphDbUrl (the database endpoint), but no explicit proxyConnection, now selects url and stores the proxy endpoint as the database URL. Please preserve the old inference with proxyConnection ?? connection.graphDbUrl != null and add a backward-compatibility test for both URLs with an absent flag.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored in df95624. transformLegacyConnection treats an absent proxyConnection as a proxy connection whenever graphDbUrl is present, so the database URL wins over the old proxy url. The AWS auth fields follow the same rule. The test with both URLs and no flag is in 77c5dea.

// The file must carry a usable endpoint in either the canonical or the
// legacy field, otherwise migration would yield an empty `graphDbUrl`.
.refine(
connection => connection.graphDbUrl != null || connection.url != null,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This accepts proxyConnection: true with only the legacy url, but migration intentionally does not use that proxy URL as the database endpoint and produces an empty graphDbUrl. The import therefore succeeds while creating an unusable connection. Could the refinement require graphDbUrl when proxyConnection is true, and add a negative import test for that inconsistent legacy shape?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaving this one as is on purpose. That shape was already broken before this PR, since a proxy connection with no graphDbUrl had no database to query. Rejecting the file would throw away the whole import, schema included, because of one missing field. Instead, the import keeps the connection with an empty URL, and the UI shows a missing database URL error (14daec7) so the user can edit the connection and fill it in. The empty-URL behavior is pinned in configuration.test.ts, and the refine comment in parseConnectionFile.ts explains it.

@kmcginnes

Copy link
Copy Markdown
Collaborator Author

@mjuarros Thanks for these. Where each one stands:

@kmcginnes
kmcginnes force-pushed the remove-sagemaker-docker-image branch from 77c5dea to 270c1e0 Compare September 26, 2026 17:00
## Considered options

- **Keep the two images.** Costs double CI build and scan time, and forces the SageMaker lifecycle script to track a separate tag lineage.
- **Keep `proxyConnection` as a permanent advanced opt-in while still unifying the image.** The relative-URL work alone unifies the image, so this was possible on its own. It costs keeping two request paths permanently, and keeping the feature gates that gave the direct path fewer capabilities than the proxy path.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be removed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Part 1 keeps the opt-in for now, so listing it as a rejected option read as a contradiction. I removed the bullet and folded its reasoning into the rationale paragraph below it, which now says the opt-in stays only as a deprecated transition. Done in 072a1f0.

Comment thread docs/guides/troubleshooting.md Outdated

### Proxy Server Cannot Be Reached

A "Connection Error" belongs here. It means the browser could not reach the Graph Explorer server, not that the server could not reach the database.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads strangely.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. It now reads: "A "Connection Error" means the browser couldn't reach the Graph Explorer server. The database itself may be fine." Done in 235d1fd.

Comment thread docs/references/configuration.md Outdated

> [!NOTE]
>
> `PUBLIC_OR_PROXY_ENDPOINT` and `USING_PROXY_SERVER` are still honored for backward compatibility and are resolved into `GRAPH_CONNECTION_URL`. An existing deployment that sets these needs no change. The default connection is a deprecated direct connection when `USING_PROXY_SERVER` is set to any value other than `true` (case-insensitive), or when `USING_PROXY_SERVER` is unset and only `PUBLIC_OR_PROXY_ENDPOINT` is provided, with no `GRAPH_CONNECTION_URL`. It connects to `PUBLIC_OR_PROXY_ENDPOINT` (or `GRAPH_CONNECTION_URL` when that is unset): the browser sends its requests to the database itself, and `IAM`, `AWS_REGION`, and `SERVICE_TYPE` are ignored. Direct connections will be removed in a future release, so switch to `GRAPH_CONNECTION_URL` and drop `PUBLIC_OR_PROXY_ENDPOINT` and `USING_PROXY_SERVER` when you can.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe PUBLIC_OR_PROXY_ENDPOINT is ignored now, correct?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite. It's only ignored when USING_PROXY_SERVER=true. Otherwise it's the URL for a deprecated direct default connection, which matches how earlier versions treated it, including when USING_PROXY_SERVER is unset. That paragraph was hard to follow, so I turned it into a list of the three cases in 9bb6181.

…ion ADR

retryFetch in the proxy server defaults to a single attempt and no
caller overrides it, so the proxy never actually retries. The direct
path's list of missing capabilities shouldn't claim otherwise.
It fires on the first call to the Graph Explorer server, which is page
load when there are no saved connections, or the first schema sync or
query otherwise, not specifically "not during schema sync."
AWS Region appears before Service Type, both only show up once AWS IAM
Auth Enabled is checked, and Service Type's options are the form's
Neptune DB / Neptune Analytics labels rather than the raw neptune-db /
neptune-graph values. Also spell out that Fetch Timeout (ms) is only
visible after checking Enable Fetch Timeout.
…ority

All three connectors send requests through it with just an endpoint
path, so a call site should never build a database URL or set proxy
headers itself.
happy-dom's Headers preserves whatever case a caller passes in, so
these tests were asserting "Content-Type" and passing for the wrong
reason; a real browser or Node lowercases it. Added a normalizeHeaders
test helper and switched the explorer tests off call-index lookups
(mock.calls[0]) since a preceding serverLogger fetch to the logger
endpoint can land at index 0, matching by URL instead.

Also drops two duplicate fetchDatabaseRequest tests: "sends a proxy
connection's request to the Graph Explorer server" duplicated "passes
the request body through to fetch", and "sends graph-db-connection-url
header when proxyConnection is absent" duplicated "sets
graph-db-connection-url header".
Renamed "marks a direct connection as deprecated" to describe what it
actually checks, and made the negative case assert the full proxied
subtitle (or that no "Direct" text renders) instead of just checking
the URL substring.
Covers editing an existing direct connection: Advanced options starts
open, unchecking the direct option immediately reveals AWS IAM Auth
Enabled again, and saving drops the proxyConnection key entirely.
Matches parseConnectionFile.ts's z.url({ protocol: /^https?$/ })
pattern and drops the now-obsolete URL.canParse comment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decouple ServerLoggerConnector from connection

2 participants