Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions .github/scripts/pg_upgrade_cluster
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
#
# pg_upgrade_cluster - Shared binary pg_upgrade mechanics for CI. Used by BOTH
# pg-upgrade-test (filesystem-installed cat_tools) and pg-tle-upgrade-test
# (pg_tle-registered cat_tools) in .github/workflows/ci.yml -- this part of
# the flow has nothing extension-specific about it, so it lives here once
# instead of being duplicated near-verbatim in both jobs.
#
# Lives under .github/, not bin/: unlike bin/test_existing (which a developer
# can run locally against a scratch database), this is CI-mechanics-only --
# pg_ctlcluster/pg_createcluster/pg_upgrade against the pgxn-tools image's
# "test" cluster convention isn't a locally-runnable workflow.
#
# USAGE: .github/scripts/pg_upgrade_cluster <subcommand> [args]
#
# recreate-old PG_VERSION
# pg-start's default "test" cluster doesn't have data checksums
# enabled, but binary pg_upgrade requires the old and new clusters to
# have MATCHING checksum/auth settings (see $INITDB_OPTS below) --
# stop, drop, and recreate the "test" cluster for PG_VERSION with them,
# then start it and wait for readiness.
#
# upgrade OLD_PG NEW_PG [EXTRA_NEW_CONF]
# Stop the old cluster, create the new cluster's "test" cluster (same
# $INITDB_OPTS), optionally append EXTRA_NEW_CONF to its
# postgresql.conf (e.g. to enable pg_tle before pg_upgrade runs), binary
# pg_upgrade OLD_PG -> NEW_PG, then start the new cluster and wait for
# readiness. On pg_upgrade failure, dumps its logs (PG17+ writes them to
# $new_datadir/pg_upgrade_output.d/; older versions write to CWD -- both
# are searched) before failing.
#
# $INITDB_OPTS (env var, required): initdb options both clusters must share so
# pg_upgrade sees consistent settings on old and new (e.g.
# "--data-checksums --auth trust"). Deliberately left unquoted at each use
# site so its (space-separated) options word-split into separate arguments.
set -euo pipefail

: "${INITDB_OPTS:?INITDB_OPTS must be set}"

recreate_old() {
local pg=$1
pg_ctlcluster "$pg" test stop
pg_dropcluster "$pg" test
# -p 5432: pg_createcluster assigns the next available port, which may not
# be 5432 after pg-start has claimed and released it. Force 5432 so
# subsequent psql/createdb calls connect without -p.
pg_createcluster -p 5432 "$pg" test -- $INITDB_OPTS
pg_ctlcluster "$pg" test start
pg_isready -t 30
}

upgrade() {
local old_pg=$1 new_pg=$2 extra_conf=${3:-}
pg_ctlcluster "$old_pg" test stop
pg_createcluster -p 5432 "$new_pg" test -- $INITDB_OPTS
# Use `if`, not `&&`: a false test under `set -e` would abort the script.
if [ -n "$extra_conf" ]; then
echo "$extra_conf" >> "/etc/postgresql/$new_pg/test/postgresql.conf"
fi
# PG17+ writes logs to $new_datadir/pg_upgrade_output.d/; older versions
# write to CWD. Search both on failure.
mkdir -p /tmp/pg_upgrade_logs
chown postgres:postgres /tmp/pg_upgrade_logs
su -c "cd /tmp/pg_upgrade_logs && /usr/lib/postgresql/$new_pg/bin/pg_upgrade \
-b /usr/lib/postgresql/$old_pg/bin \
-B /usr/lib/postgresql/$new_pg/bin \
-d /var/lib/postgresql/$old_pg/test \
-D /var/lib/postgresql/$new_pg/test \
-o '-c config_file=/etc/postgresql/$old_pg/test/postgresql.conf' \
-O '-c config_file=/etc/postgresql/$new_pg/test/postgresql.conf'" postgres \
|| { find /tmp/pg_upgrade_logs \
"/var/lib/postgresql/$new_pg/test/pg_upgrade_output.d" \
-name '*.log' 2>/dev/null | sort | xargs -r tail -n +1; exit 1; }
pg_ctlcluster "$new_pg" test start
pg_isready -t 30
}

usage() {
echo "usage: .github/scripts/pg_upgrade_cluster <subcommand> [args]" >&2
echo " recreate-old PG_VERSION" >&2
echo " upgrade OLD_PG NEW_PG [EXTRA_NEW_CONF]" >&2
exit 2
}

main() {
local cmd=${1:-}
shift || true
case "$cmd" in
recreate-old) recreate_old "$@" ;;
upgrade) upgrade "$@" ;;
*) usage ;;
esac
}

main "$@"
71 changes: 71 additions & 0 deletions .github/workflows/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# CI Instructions

## Reducing CI wall-clock time

Don't chase this proactively -- it's not worth adding real complexity for. But if a *simple*
latency win is available while touching a job anyway, take it. Facts and constraints to reason
from:

- **GitHub Actions steps within one job are strictly sequential.** There is no native
"run step A and step B concurrently" mechanism. The only real unit of parallelism is the
*job* (already exploited here via the `pg`/matrix-leg lists) -- reordering two independent
steps changes nothing about total wall-clock time unless one of them is actually
backgrounded (`cmd &` ... `wait`) within a single step's shell script.
- **`pg-start VERSION` is not a no-op or a "wait for something already running."** It installs
that PostgreSQL major version via `apt.postgresql.org.sh` (network + package install) and
then creates and starts the "test" cluster (`pg_createcluster --start`) -- both genuinely
slow, and both block until done, like every `run:` step.
- **Backgrounding a step next to `pg-start` (or any `apt-get install`) is NOT safe by
default**: apt/dpkg serialize on a lock file, so a second concurrent `apt-get install` would
either fail immediately or block waiting for the lock -- effectively serializing anyway, but
now with added flakiness risk. Only background work that doesn't touch the package manager
(e.g. a `git clone`, or a build step that's pure `gmake`/`gcc` against already-installed
headers) alongside an apt-based step.
- **A genuine, low-complexity candidate, if this is ever worth doing**: `pg-tle-test`'s
"Install pgtap" step (`make pgtap`, which builds pgTAP via `gmake`/`gcc`, not apt) and its
"Build and install pg_tle" step (`git clone` + `make`, also not apt) are mutually
independent -- neither depends on the other's output. Backgrounding one (`make pgtap &`)
while the other runs, then `wait`ing for it before the step that needs pgtap, would overlap
two of the job's slower operations. This is a small, contained change (one background job,
one `wait`, one exit-code check) -- not the kind of complexity worth avoiding on principle.
- **`actions/cache` is likely a better lever than backgrounding.** pg_tle is rebuilt from
source (git clone + C compile) fresh in every `pg-tle-test` matrix entry and on both the old
and new cluster in every `pg-tle-upgrade-test` leg -- the same pg_tle version, rebuilt
repeatedly across runs with nothing changed. Caching the built `pg_tle.so` + control/SQL
files (keyed on PG major version + `PGTLE_VERSION`) would skip the clone+compile entirely on
a cache hit, without touching the job's logic at all. Worth trying before hand-rolled
parallelism if CI time on the pg_tle jobs becomes a real pain point.
- **Double-run check**: this repo's `push:` trigger is scoped to `branches: [master]` (not a
bare `on: [push, pull_request]`), so pushes to a feature/PR branch only fire the
`pull_request` event, not both -- confirmed via run history, no double-run here. If this
workflow's triggers are ever changed, re-verify that scoping is preserved; an unscoped
`push:` trigger would double-run CI on every PR-branch commit.

## Docs-only pushes and "what was actually tested"

A docs-only push makes the heavy jobs report "skipped" for THAT commit, which is correct (the
code didn't change) but easy to misread on the PR's Checks tab as "never tested." The `changes`
job's "Find the last commit where real code changed" step handles this: it walks backward
through history to the newest commit that actually changed code, looks up that commit's CI run,
and `all-checks-passed` reports it (link + green/red) in its step summary -- or explicitly notes
when the entire PR has never touched anything but docs. Don't assume "skipped" means "untested"
without checking that summary first.

## `pull_request_target` workflows can't be tested from the PR that changes them

GitHub always executes the workflow FILE for a `pull_request_target` event from the BASE
branch (master), never the PR's own version -- a deliberate security control so a fork PR can't
alter its own reviewer's permissions/behavior. `claude-code-review.yml` runs this way. Practical
consequence: a PR that changes that file (e.g. adjusting its `permissions:` block) cannot prove
the change took effect by watching that PR's own `claude-review` check -- it's still running
master's old version. Verify changes to that file only after merging to master, on the next PR.

## Where CI-only shell logic lives

`.github/scripts/` holds shell scripts that are **only** meaningful inside a CI job (e.g.
`pg_upgrade_cluster`, which drives `pg_ctlcluster`/`pg_createcluster`/`pg_upgrade` against the
pgxn-tools image's "test" cluster convention -- not something a developer would run locally).
This is distinct from the repo-root `bin/` scripts (`bin/test_existing`, `bin/assert_fs_clean`),
which a developer CAN run locally against a scratch database and are also usable outside CI.
When adding a new shared CI shell helper, put it in `.github/scripts/` if it's CI-only, `bin/`
if it's usable locally too.
Loading
Loading