Port selection is check-then-bind: concurrent JVMs collide, and a lost race can silently adopt another Hoverfly instance

[devin-ai-integration]

Summary

Hoverfly.start() picks its proxy/admin ports in the JVM, but the ports are bound by the hoverfly binary in a separate process. Between the check and the bind there is a window in which any other process can take the port. When it does, the failure mode is not just a hard error — the Java client can decide it started successfully while actually talking to another Hoverfly instance.

We hit this in CI, where multiple test JVMs on one host start Hoverfly concurrently (Bazel test targets). Symptoms:

java.lang.IllegalStateException: Port is already in use: 54726
Failed to set mode: Unexpected response (code=502)

Version: io.specto:hoverfly-java:0.19.0 (code below from tag 0.19.0).

Why it races

Hoverfly.start() → HoverflyUtils.checkPortInUse:

static void checkPortInUse(int port) {
    try (final ServerSocket ignored = new ServerSocket(port, 1, InetAddress.getLoopbackAddress())) {
        // Do nothing
    } catch (IOException e) {
        throw new IllegalStateException("Port is already in use: " + port);
    }
}

The check socket is closed before the process starts; the actual bind happens later in the child process:

commands.add("-pp"); commands.add(String.valueOf(hoverflyConfig.getProxyPort()));
commands.add("-ap"); commands.add(String.valueOf(hoverflyConfig.getAdminPort()));
startedProcess = new ProcessExecutor().command(commands).start();

So the sequence is check → close → exec → bind, and the port is unowned for the whole middle part. Letting hoverfly-java pick a free port itself does not help: port 0 is also resolved Java-side by opening a ServerSocket(0), reading getLocalPort(), and closing it, so the same window exists. There is no way to make this atomic from the Java side, because the process that performs the final bind is not the one that selected the port.

Why losing the race can be silent

After exec, readiness is decided by polling /health on the configured admin port — not by checking that the child process is alive or that it owns that port. If our binary died on bind because a neighbouring Hoverfly already owns the admin port, that neighbour answers the health check and start() returns as if the local instance were healthy. From then on:

• setMode / simulate hit the other instance's admin API — either failing oddly (the 502 above, when the neighbour is mid-startup) or, worse, silently mutating the neighbour's mode/simulation;
• proxied traffic goes through the neighbour's proxy port, so a test can record or replay against the wrong simulation instead of failing.

setSimulation failures are logged rather than thrown in some paths, which makes the wrong-instance case even harder to notice.

Repro shape

Start two JVMs concurrently on one host, each doing new Hoverfly(localConfigs(), SIMULATE).start() with independently chosen free ports, in a loop. Under load you get either Port is already in use: <port> or a client that reports healthy while pointing at the other JVM's Hoverfly.

Suggested fix

The port selection must be done by whoever binds:

1. Pass -pp 0 -ap 0 through to the binary and have hoverfly report the ports the OS actually assigned (log lines or a small machine-readable line on stdout), then have hoverfly-java parse them back and use them for the admin client and proxy configuration. Today the binary just echoes the requested value, so there is nothing to read back:

   $ hoverfly -pp 0 -ap 0
   Default proxy port has been overwritten           port=0
   Default admin port has been overwritten           port=0
   Admin interface is starting...                    AdminPort=0
   serving proxy
   

2. Independently of that, start() should verify the instance it health-checked is the one it launched (e.g. fail if the child process has exited, and/or match an identity exposed by the admin API) so a foreign Hoverfly can never be adopted silently.

(2) alone would already turn today's silent cross-talk into a clean failure; (1) removes the race entirely.

Workaround for others hitting this

We serialize allocate-and-start across test JVMs with a host-wide lock, then verify ownership by installing a unique simulation via the admin port and requesting it back through the proxy port, retrying on fresh ports if the answer doesn't match. That is a mitigation, not a fix — an unrelated process can still take the port after selection.