Fail CI when the checked-in test clients drift

.github/workflows/ci.yml

@@ -39,3 +39,32 @@ jobs:
 
       - name: Test
         run: dotnet test Linq2GraphQL.CI.slnf --no-build --verbosity normal
+
+  generated-clients:
+
+    name: Test clients up to date
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup .NET
+        uses: actions/setup-dotnet@v4
+        with:
+          dotnet-version: 10.0.x
+          cache: true
+          cache-dependency-path: '**/packages.lock.json'
+
+      - name: Restore tools
+        run: dotnet tool restore
+
+      - name: Restore
+        run: dotnet restore Linq2GraphQL.CI.slnf --locked-mode
+
+      # Regenerates test/Linq2GraphQL.TestClient* from the test servers' live
+      # schemas and fails if the result differs from what is committed.
+      - name: Check checked-in test clients
+        shell: pwsh
+        run: ./scripts/regenerate-test-clients.ps1 -Check

CLAUDE.md

@@ -21,6 +21,12 @@ dotnet restore Linq2GraphQL.CI.slnf --locked-mode
 dotnet build   Linq2GraphQL.CI.slnf --no-restore
 dotnet test    Linq2GraphQL.CI.slnf --no-build
 
+# Also CI: verify the checked-in test clients still match the generator
+./scripts/regenerate-test-clients.ps1 -Check
+
+# Refresh them after a template or test-schema change
+./scripts/regenerate-test-clients.ps1
+
 # Single test / class
 dotnet test test/Linq2GraphQL.Tests --filter "FullyQualifiedName~QueryTests.Hello_WithNoName_HelloWorld"
 dotnet test test/Linq2GraphQL.Tests --filter "FullyQualifiedName~ExpressionParserTests"
@@ -63,6 +69,6 @@ Key details that bite:
 
 `Linq2GraphQL.Tests` is xUnit + Shouldly + Moq. It spins up the real GraphQL server in-process with `WebApplicationFactory<Program>` — `Linq2GraphQL.TestServer` is HotChocolate over the POCOs in `TestServer.Shared`, and `TestServerNullable` is the same schema for the nullable client. `SampleClientFixture` / `SampleClientNullableFixture` wire the generated client to that in-memory host (safe mode on, SSE subscriptions), and test classes take them via `IClassFixture<>`. So most tests are end-to-end: an assertion failure can come from the expression parser, the query text, or the server's own resolvers.
 
-`Linq2GraphQL.TestClient` / `TestClientNullable` hold **checked-in generated output**. They are not regenerated by the build — when a template or schema changes, run the generator against a locally running TestServer and commit the result, then verify the round trip still passes.
+`Linq2GraphQL.TestClient` / `TestClientNullable` hold **checked-in generated output**. They are not regenerated by the build — run `./scripts/regenerate-test-clients.ps1` (it boots both test servers over plain HTTP, regenerates both clients with the flags that produced the committed output, and writes them back) and commit the diff. The `generated-clients` CI job runs the same script with `-Check` and fails if the committed output no longer matches the generator, so a template or schema change that you forget to regenerate is caught in CI rather than passing against stale output.
 
 `docs/`, `docs/StarWars.Client` and `StartGG/` are the Blazor documentation site and sample clients; they are outside the CI solution filter and can drift.

REPO_REPORT.md

@@ -53,7 +53,7 @@ Three stages, and understanding them is usually the whole job:
 ## 5. Highlights — the risks
 
 - ~~**T4 is the sharpest edge in the repo.**~~ **Fixed.** Templates are now preprocessed at build time by the pinned `dotnet-t4` local tool into gitignored `X.g.cs` files, so a `.tt` edit takes effect on the next `dotnet build` on any platform and stale template logic cannot be built. (Switching it on revealed one already-dormant edit: `ScalarTemplate.tt` referenced a non-existent `GraphqlType.ScalarTypeName` and had never been regenerated.)
-- **Generated test clients are checked in but never regenerated by the build.** `TestClient`/`TestClientNullable` must be re-generated by hand against a locally running TestServer after any template or schema change. Nothing in CI detects the drift — the tests keep passing against stale output.
+- ~~**Generated test clients are checked in but never regenerated by the build.**~~ **Fixed.** `./scripts/regenerate-test-clients.ps1` boots both test servers over plain HTTP and regenerates `TestClient`/`TestClientNullable`, and the `generated-clients` CI job runs it with `-Check` so committed output that no longer matches the generator fails the build. Refreshing the output picked up three pieces of accumulated drift: the newly active template edits, the signed `Byte` scalar mapping (`byte` → `sbyte`), and the `raiseError`/`raiseAuthError` query methods.
 - **Coupling between alias hashing and deserialization.** `Utilities.GetArgumentsId` must produce the *same* hash at write time and read time. Any change to argument hashing breaks reads as well as writes, and the failure mode is a silently missing field rather than an exception.
 - **Ambient static generator state.** `GeneratorSettings.Current` is read from inside templates, so nullable/non-nullable output depends on global mutable state rather than a passed parameter. Fine today; awkward if generation ever needs to run concurrently.
 - **`docs/` and `StartGG/` are outside the CI filter and can drift.** They are in `Linq2GraphQL.sln` but not `Linq2GraphQL.CI.slnf`, so nothing verifies they still compile — roughly 15k lines CI never touches.
@@ -70,7 +70,7 @@ Three stages, and understanding them is usually the whole job:
 ## 7. If I were picking up work here
 
 1. ~~Wire up **CLI T4 regeneration**~~ — done: `dotnet-t4` runs from the generator's csproj on every build, and the preprocessed output is no longer checked in.
-2. Add a **CI job that regenerates the test clients and diffs them** — it closes the stale-generated-output gap for free.
+2. ~~Add a **CI job that regenerates the test clients and diffs them**~~ — done: `scripts/regenerate-test-clients.ps1 -Check`, run by the `generated-clients` job.
 3. Pull `docs/` and `StartGG/` into a **build-only CI job** so they cannot rot unnoticed.
 4. Get the **WebSocket subscription transport under test**, even against a standalone host outside `WebApplicationFactory`.
 5. Delete `nuget.config.backup` and the stray root `StarWars.Client/`.

scripts/regenerate-test-clients.ps1

@@ -0,0 +1,148 @@
+#!/usr/bin/env pwsh
+<#
+.SYNOPSIS
+    Regenerates the checked-in test clients from the test servers' live schemas.
+
+.DESCRIPTION
+    test/Linq2GraphQL.TestClient and test/Linq2GraphQL.TestClientNullable hold
+    generated output that the test suite compiles against. Nothing in the normal
+    build refreshes it, so it silently drifts whenever a template or the test
+    schema changes.
+
+    This script boots each test server over plain HTTP, runs the generator
+    against it with the flags that produced the checked-in output, and writes
+    the result back over the Generated folders. CI runs the same script and
+    fails if the result differs from what is committed.
+
+.PARAMETER Check
+    Do not keep the regenerated output: restore the working tree afterwards and
+    exit non-zero if anything differed. Used by CI.
+
+.EXAMPLE
+    ./scripts/regenerate-test-clients.ps1
+    ./scripts/regenerate-test-clients.ps1 -Check
+#>
+[CmdletBinding()]
+param(
+    [switch]$Check
+)
+
+$ErrorActionPreference = 'Stop'
+$repoRoot = Split-Path -Parent $PSScriptRoot
+
+$clients = @(
+    @{
+        Name      = 'SampleClient'
+        Server    = 'test/Linq2GraphQL.TestServer'
+        Output    = 'test/Linq2GraphQL.TestClient/Generated'
+        Namespace = 'Linq2GraphQL.TestClient'
+        Port      = 5180
+        Nullable  = $false
+    }
+    @{
+        Name      = 'SampleNullableClient'
+        Server    = 'test/Linq2GraphQL.TestServerNullable'
+        Output    = 'test/Linq2GraphQL.TestClientNullable/Generated'
+        Namespace = 'Linq2GraphQL.TestClientNullable'
+        Port      = 5181
+        Nullable  = $true
+    }
+)
+
+function Wait-ForServer([string]$url, [System.Diagnostics.Process]$process) {
+    foreach ($attempt in 1..60) {
+        if ($process.HasExited) {
+            throw "Server exited with code $($process.ExitCode) before becoming ready."
+        }
+
+        try {
+            Invoke-WebRequest -Uri $url -Method Get -TimeoutSec 2 -SkipHttpErrorCheck | Out-Null
+            return
+        }
+        catch {
+            Start-Sleep -Milliseconds 500
+        }
+    }
+
+    throw "Server at $url did not become ready in 30 seconds."
+}
+
+Push-Location $repoRoot
+try {
+    Write-Host '==> Building generator and test servers' -ForegroundColor Cyan
+    dotnet build Linq2GraphQL.CI.slnf --nologo
+    if ($LASTEXITCODE -ne 0) { throw 'Build failed.' }
+
+    foreach ($client in $clients) {
+        $endpoint = "http://127.0.0.1:$($client.Port)/graphql/"
+        Write-Host "==> Starting $($client.Server) on $endpoint" -ForegroundColor Cyan
+
+        $server = Start-Process -PassThru -NoNewWindow -FilePath 'dotnet' -ArgumentList @(
+            'run', '--project', $client.Server, '--no-build', '--no-launch-profile'
+        ) -Environment @{
+            ASPNETCORE_URLS        = "http://127.0.0.1:$($client.Port)"
+            ASPNETCORE_ENVIRONMENT = "Development"
+        }
+
+        try {
+            Wait-ForServer -url $endpoint -process $server
+
+            Write-Host "==> Generating $($client.Name) into $($client.Output)" -ForegroundColor Cyan
+            $arguments = @(
+                'run', '--project', 'src/Linq2GraphQL.Generator', '--no-build', '--no-launch-profile', '--'
+                $endpoint
+                "-c=$($client.Name)"
+                "-n=$($client.Namespace)"
+                "-o=$(Join-Path $repoRoot $client.Output)"
+                '-s=true'
+                '-d=true'
+            )
+            if ($client.Nullable) { $arguments += '-nu=true' }
+
+            $sentinel = Join-Path $repoRoot "$($client.Output)/Client/$($client.Name).cs"
+            $generatedAt = Get-Date
+
+            dotnet @arguments
+
+            # The generator reports schema failures on stdout without a non-zero
+            # exit code, so verify it actually wrote fresh output.
+            if (-not (Test-Path $sentinel) -or (Get-Item $sentinel).LastWriteTime -lt $generatedAt) {
+                throw "Generating $($client.Name) failed - $sentinel was not written."
+            }
+        }
+        finally {
+            if (-not $server.HasExited) {
+                $server.Kill($true)
+            }
+            $server.WaitForExit()
+        }
+    }
+
+    if (-not $Check) {
+        Write-Host '==> Done. Review and commit the regenerated output.' -ForegroundColor Green
+        exit 0
+    }
+
+    $paths = $clients | ForEach-Object { $_.Output }
+
+    # git status refreshes the index first, so files rewritten with identical
+    # content are not reported as changed just because their mtime moved.
+    $drifted = @(git status --porcelain -- @paths)
+    $diff = git diff --stat -- @paths
+
+    git checkout -- @paths
+
+    if ($drifted.Count -gt 0) {
+        Write-Host ''
+        Write-Host 'The checked-in test clients do not match the generator output:' -ForegroundColor Red
+        Write-Host ($diff -join [Environment]::NewLine)
+        Write-Host ''
+        Write-Host 'Run ./scripts/regenerate-test-clients.ps1 and commit the result.' -ForegroundColor Red
+        exit 1
+    }
+
+    Write-Host '==> Checked-in test clients match the generator output.' -ForegroundColor Green
+}
+finally {
+    Pop-Location
+}

test/Linq2GraphQL.TestClient/Generated/Client/MutationMethods.cs

@@ -5,39 +5,56 @@
 // Url: https://linq2graphql.com
 //---------------------------------------------------------------------
 
-using System.Collections.Generic;
 using System;
+using System.Collections.Generic;
 using Linq2GraphQL.Client;
 
 namespace Linq2GraphQL.TestClient;
 
+/// <summary>
+/// Implementation of MutationMethods GraphQL operations
+/// </summary>
 public class MutationMethods : IMutationMethods
 {
     private readonly GraphClient client;
 
+    /// <summary>
+    /// Initializes a new instance of the MutationMethods class
+    /// </summary>
+    /// <param name="client">The GraphQL client instance</param>
     public MutationMethods(GraphClient client)
     {
-        this.client = client;
+        this.client = client ?? throw new ArgumentNullException(nameof(client));
     }
 
+    /// <summary>
+    /// Executes setName GraphQL operation
+    /// </summary>
+    /// <param name="setName">The operation parameters</param>
+    /// <returns>GraphQL query result of type GraphQuery<string></returns>
     public GraphQuery<string> SetName(string name = null)
     {
-	    var arguments = new List<ArgumentValue>
+        var arguments = new List<ArgumentValue>
         {
-    	    new("name","String", name),
+            new("name", "String", name),
         };
 
-        return new GraphQuery<string>(client,  "setName", OperationType.Mutation, arguments); 
+        return new GraphQuery<string>(client, "setName", OperationType.Mutation, arguments);
     }
 
+    /// <summary>
+    /// Executes addCustomer GraphQL operation
+    /// </summary>
+    /// <param name="addCustomer">The operation parameters</param>
+    /// <returns>GraphQL query result of type GraphQuery<Customer></returns>
     public GraphQuery<Customer> AddCustomer(CustomerInput customer = null)
     {
-	    var arguments = new List<ArgumentValue>
+        var arguments = new List<ArgumentValue>
         {
-    	    new("customer","CustomerInput", customer),
+            new("customer", "CustomerInput", customer),
         };
 
-        return new GraphQuery<Customer>(client,  "addCustomer", OperationType.Mutation, arguments); 
+        return new GraphQuery<Customer>(client, "addCustomer", OperationType.Mutation, arguments);
     }
 
-    }
+}