Skip to content

Build tdigest for Ubuntu 20.04 (focal) — PG11–16 pipeline - #1239

Open
Buket (aybuket) wants to merge 5 commits into
developfrom
tdigest-focal
Open

Buket (aybuket) wants to merge 5 commits into
developfrom
tdigest-focal

Conversation

@aybuket

@aybuket Buket (aybuket) commented Sep 10, 2026 •

Copy link
Copy Markdown

What and why

PGDG no longer publishes packages for Ubuntu 20.04 (focal-pgdg returns 404), and its archive is frozen. This pipeline rebuilds the latest stable upstream tdigest release for PostgreSQL 11–16, including matching debug-symbol packages, using the archived focal Debian packaging and build tooling.

Release and packaging discovery

  • tdigest_version defaults to latest. The workflow resolves GitHub's latest stable release once and passes the resolved version, source URL, archive format, and SHA-256 into the builder. The smoke-test job uses that same resolved version.
  • Automatic builds download the upstream release ZIP and verify it against the SHA-256 digest published by GitHub. Missing assets, missing digests, and checksum mismatches fail the build.
  • Explicit versions remain supported. For historical releases without a checksummed release ZIP, callers can provide the SHA-256 for the GitHub tag tarball.
  • APT selects the archived tdigest source package used for debian/. Upgrade-test baselines are discovered from APT separately for each PostgreSQL major, before installing the newly built packages.
  • There are no hard-coded tdigest release versions or checksums in the pipeline. Resolved source details and the selected Debian packaging source version are saved in tdigest-source.env alongside the artifacts.

Implementation

File Role
dockerfiles/focal-tdigest-builder/Dockerfile Ubuntu 20.04 builder with archived PGDG tooling and release-discovery/archive utilities
scripts/resolve_tdigest_focal Validates PostgreSQL majors and resolves the upstream release, source URL, and checksum
scripts/build_tdigest_focal Combines upstream source with archived Debian packaging, builds packages, and validates debug symbols
scripts/smoke_test_focal_tdigest_debs Verifies clean installation and upgrades in a stock focal container
.github/workflows/build-tdigest-focal.yml Orchestrates discovery, build, signing, artifact checks, and smoke tests

One multi-version PGXS source build covers all requested majors. The builder writes debian/pgversions and regenerates debian/control with pg_buildext updatecontrol, so only the requested packages are declared and built. The default is 11 12 13 14 15 16; values outside that range are rejected before the build.

Each major produces:

  • postgresql-<major>-tdigest, containing its own library, extension control file, and SQL upgrade chain.
  • postgresql-<major>-tdigest-dbgsym, containing matching debug information.

The build enables debug information and requires exactly one debug-symbol package per major. Ubuntu .ddeb artifacts are renamed to .deb so runtime and symbol packages follow the same signing and artifact-upload path.

Archived packaging and upstream source are extracted into separate directories within a fresh build workspace. This fixes the source-directory collision when rebuilding the same upstream version as the archived packaging. Explicit historical rebuilds also handle Debian changelog version ordering.

Verification performed by the pipeline

  • Source integrity: verify the upstream archive checksum; obtain archived Debian packaging through APT's signed repository metadata.
  • Package signing: sign every runtime and debug-symbol .deb with the existing debsigner image and require an embedded _gpgmaint signature.
  • Self-contained runtime: require the extension control file and SQL reaching the resolved upstream version.
  • Debug symbols: require an exact runtime-version dependency and actual DWARF debug information matching the runtime library's ELF build ID.
  • Clean install: install runtime and symbol packages, confirm both remain installed at the expected package versions, create the extension, check extversion, and execute a percentile aggregate on every major.
  • Upgrade: install the dynamically selected archived runtime and symbol packages, verify the extension baseline, upgrade both packages, run ALTER EXTENSION tdigest UPDATE, and repeat version and aggregate checks.
  • Optional upstream regressions: run_tests=1 runs the upstream regression suite through the multi-version packaging build.

Local validation

Tested using Podman with Ubuntu 20.04 containers:

  • Built all 12 runtime/debug-symbol packages for PG11–16 and passed dependency, ELF build-ID, and DWARF checks.
  • Passed 41 upstream regression tests on each major: 246 tests total.
  • Passed clean-install and archived-package upgrade smoke tests on all six majors.
  • After removing the remaining archive-version pins, rebuilt all 12 packages and reran the smoke tests with dynamically discovered APT baselines.
  • Verified a historical rebuild matching the archived upstream version for PG11 and PG16.
  • Passed resolver input checks, Bash syntax checks, ShellCheck, actionlint, YAML parsing, and git diff --check.

Package signing was not exercised locally; it requires the repository's GitHub Actions secrets.

Workflow inputs

Input Default Behavior
tdigest_version latest Blank or latest discovers the latest stable release; an explicit version selects that release
pg_versions 11 12 13 14 15 16 Space-separated subset of supported PostgreSQL majors
tdigest_sha256 blank Blank uses the release ZIP and GitHub-published digest; an explicit checksum selects tag-tarball verification
run_tests 0 Set to 1 to run upstream regression tests

The workflow supports manual dispatch and runs on pushes to tdigest-focal. The package revision defaults to 1.citus20.04+1; the upstream part of the package version comes from release discovery.

Rebuild tdigest .deb packages for Ubuntu 20.04 (focal) from upstream source, mirroring the focal PostGIS pipeline. PGDG dropped focal (focal-pgdg 404s, frozen archive stops at tdigest 1.4.3), so newer tdigest (default 1.4.5, memory-safety fixes) is rebuilt from upstream source + the frozen focal-era debian/ packaging. One multi-version source build emits postgresql-<major>-tdigest for PG12-16, signed via the debsigner image and smoke-tested in a stock ubuntu:20.04.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Some advertised dispatch inputs fail because of a source-directory collision and unsupported PostgreSQL majors.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Builds, signs, and smoke-tests tdigest 1.4.6 packages for PostgreSQL 11–16 on Ubuntu 20.04.

Changes:

  • Adds a focal builder using archived PGDG packaging.
  • Adds package signing and validation workflow.
  • Tests clean installation and upgrades from tdigest 1.4.3.
File summaries
File Description
scripts/build_tdigest_focal Builds multi-version tdigest packages.
scripts/smoke_test_focal_tdigest_debs Tests installation and upgrades.
dockerfiles/focal-tdigest-builder/Dockerfile Defines the focal build environment.
.github/workflows/build-tdigest-focal.yml Orchestrates build, signing, and testing.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/build-tdigest-focal.yml
Comment thread scripts/build_tdigest_focal Outdated
Resolve the upstream release and checksum once per workflow, and discover archived packaging and upgrade baselines through APT.

Validate PG11-16 inputs, isolate packaging sources, require matching DWARF symbol packages, and strengthen clean-install and upgrade checks.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Historical rebuild handling and automatic patch removal can produce unusable or incomplete packages.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/build_tdigest_focal:66

  • The fresh workspace does not prevent stale files in the persistent OUTPUT_DIR. Reusing the documented /packages volume after building another version or set of majors leaves extra .deb files that the signing, upload, verification, and smoke-test globs will include. Remove the previous tdigest artifacts before collecting this invocation's output.

dockerfiles/focal-tdigest-builder/Dockerfile:62

  • This line does not enforce the debhelper >= 13 requirement stated above it: dpkg-query only prints installed versions and succeeds for any installed debhelper. Make the image check compare the installed version so an incompatible archive/toolchain fails at this intended guard rather than later during package dependency resolution.
RUN dpkg-query -W -f='${Package} ${Version}\n' debhelper postgresql-common-dev dh-exec
  • Files reviewed: 5/5 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread scripts/build_tdigest_focal Outdated
Comment thread scripts/build_tdigest_focal
Comment thread scripts/smoke_test_focal_tdigest_debs
Apply the full Debian patch series and enforce debhelper >= 13. Stage and validate artifacts before replacing previous tdigest output.

Derive upgradeable revisions for same-upstream rebuilds and select compatible archived runtime/dbgsym baselines without attempting extension downgrades.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants