diff --git a/.gitignore b/.gitignore index c2d78e8..8d5a10f 100644 --- a/.gitignore +++ b/.gitignore @@ -9,3 +9,14 @@ out/ *.iso *.qcow2 *.raw + +# Let users drop their OWN hosts into hosts/ without git noise: ignore the +# whole hosts/ directory, then un-ignore only the hosts we manage centrally +# (the ones we ship + write to). A user-added hosts// stays +# untracked; our managed hosts remain version-controlled as usual. +hosts/* +!hosts/_appliance-disk/ +!hosts/_appliance_iso/ +!hosts/_installer-iso/ +!hosts/coder-thinkcentre/ +!hosts/qemu-arm64/ diff --git a/Makefile b/Makefile index 7a76e67..ab207f4 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -# Coder box — appliance image build targets. +# Coder box — image build targets. # # An "appliance" is the box prebuilt as a bootable image (no install.sh): # it boots straight into the fully-configured Coder box. Three formats: @@ -7,12 +7,19 @@ # make appliance/qcow2 # disk image (persistent; boots in QEMU/libvirt) # make appliance/raw # disk image (persistent; dd-able to a drive) # -# Each format also takes an architecture suffix; short names are normalized to +# The "installer" is the box as an ISO that will install coder/box onto real +# hardware. For now it boots the same full GUI box as the appliance ISO; ISO only +# (no disk images): +# +# make installer/iso +# +# Each target also takes an architecture suffix; short names are normalized to # a *-linux triple (e.g. aarch64 -> aarch64-linux): # # make appliance/iso/x86_64-linux # make appliance/qcow2/aarch64-linux # make appliance/raw/aarch64 +# make installer/iso/aarch64-linux # # Requires Nix with flakes enabled (nix-command + flakes). All builds run on # Linux only; cross-arch builds need a matching builder (native remote builder @@ -26,6 +33,14 @@ NIX ?= nix FLAKE ?= . +# Build revision injected into images (installer boot menu, /etc/coder-box-rev). +# We build through a path flakeref (getFlake (toString ./.)), which carries no +# git metadata, so self.rev/dirtyRev are empty — compute the rev here and pass +# it via the installer's `coderBox.rev` option. Full commit hash, with a -dirty +# suffix when the working tree has uncommitted changes. Empty if not a git +# checkout (the module then falls back to self.rev / "unknown"). +GIT_REV := $(shell git rev-parse HEAD 2>/dev/null)$(shell git diff-index --quiet HEAD -- 2>/dev/null || echo -dirty) + # Normalize an arch token to a *-linux triple: $(call norm_arch,aarch64) -> aarch64-linux norm_arch = $(if $(filter %-linux,$(1)),$(1),$(1)-linux) @@ -52,10 +67,18 @@ define box_build @mkdir -p out $(NIX) build --impure --no-write-lock-file --print-out-paths \ --out-link 'out/$(subst /,-,$@)' --expr \ - 'let f = builtins.getFlake (toString ./.); in (f.nixosConfigurations.$(1).extendModules { modules = [ { nixpkgs.hostPlatform = "$(if $(4),$(call norm_arch,$(4)),$${builtins.currentSystem})"; $(3) } ]; }).config.system.build.$(2)' + 'let f = builtins.getFlake (toString ./.); in (f.nixosConfigurations.$(1).extendModules { modules = [ { nixpkgs.hostPlatform = "$(if $(4),$(call norm_arch,$(4)),$${builtins.currentSystem})"; coderBox.rev = "$(GIT_REV)"; $(3) } ]; }).config.system.build.$(2)' endef -.PHONY: appliance/iso appliance/qcow2 appliance/raw +.PHONY: installer/iso appliance/iso appliance/qcow2 appliance/raw + +# installer/iso is listed first so it's the default goal (bare `make`). + +# ── installer/iso — installer ISO (hosts/_installer-iso); ISO only ──────────── +installer/iso: + $(call box_build,_installer-iso,isoImage,,) +installer/iso/%: + $(call box_build,_installer-iso,isoImage,,$*) # ── appliance/iso — ephemeral appliance ISO (hosts/_appliance_iso) ─────────── appliance/iso: diff --git a/README.md b/README.md index 441ec11..e80788d 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ NixOS configuration for Coder demo and workshop boxes. flake.nix # entry point: nixosConfigurations. per machine flake.lock # pinned nixpkgs / disko / nixos-facter-modules configuration.nix # shared NixOS config (all machines) -Makefile # appliance build targets: appliance/{iso,qcow2,raw}[/] +Makefile # image build targets: appliance/{iso,qcow2,raw}, installer/iso [/] local.nix.example # template copied to hosts//local.nix by install.sh .gitignore # ignores hosts/*/local.nix install.sh # one-shot installer: disko + nixos-install + bake /etc/nixos-repo @@ -39,9 +39,15 @@ nixos/ k3s-sysbox.nix # k3s + sysbox-runc runtime class k3s-podman.nix # k3s + rootless Podman socket screenconnect.nix # optional ScreenConnect remote access client - _appliance/ # prebuilt-appliance modules (ISO + persistent disk) - box-turnkey.nix # shared turn-key bits for appliances (login + Coder bootstrap) - live-iso.nix # ephemeral appliance ISO module (hosts/_appliance_iso) + _images/ # prebuilt-image modules (appliance + installer) + box-turnkey.nix # shared turn-key Coder box (login + Coder bootstrap); all image hosts + base/ # primitives shared by every image + hardware.nix # all-hardware (boot on arbitrary hardware) + iso.nix # ISO mechanics (iso-image.nix, EFI/BIOS/USB bootable, bootloader) + appliance/ + iso.nix # appliance ISO module (hosts/_appliance_iso) + installer/ + iso.nix # installer ISO module (hosts/_installer-iso) pkgs/ coder.nix # custom Coder server package coderd-provider.nix # terraform-provider-coderd package @@ -54,9 +60,11 @@ hosts/ templates/ nook-android/ # Workspace: build trmnl-nook-simple-touch APK _appliance_iso/ # `_appliance_iso` host: ephemeral appliance ISO (no disk install) - default.nix # imports nixos/_appliance/live-iso.nix (no disko/facter/hardware-config) + default.nix # imports nixos/_images/appliance/iso.nix (no disko/facter/hardware-config) _appliance-disk/ # `_appliance-disk` host: persistent qcow2/raw disk image - default.nix # imports disko-standard.nix + nixos/_appliance/box-turnkey.nix + default.nix # imports disko-standard.nix + nixos/_images/box-turnkey.nix + _installer-iso/ # `_installer-iso` host: installer ISO (ISO only; installs box onto hardware) + default.nix # imports nixos/_images/installer/iso.nix coderd/ main.tf # manages all Coder templates via coderd Terraform provider templates/ @@ -74,7 +82,7 @@ right config on the running box. Adding a new host means creating a host folder, no flake.nix edit. The installer does this for you. Hosts whose folder name starts with an underscore (`_appliance_iso`, -`_appliance-disk`) are image/appliance builds, not per-machine installs: they +`_appliance-disk`, `_installer-iso`) are image builds, not per-machine installs: they do **not** get the folder-name hostname and instead inherit the central default `networking.hostName = "coder-box"` (set in `configuration.nix`). @@ -155,8 +163,8 @@ image at `out/appliance-raw/coder-box-appliance-*.raw` (or `out/appliance-qcow2/coder-box-appliance-*.qcow2`). All names carry the arch, e.g. `coder-box-appliance-aarch64-linux.iso`. -The turn-key login + Coder admin bootstrap shared by both flavours live in -[`nixos/_appliance/box-turnkey.nix`](nixos/_appliance/box-turnkey.nix): autologin to the `coderbox` +The turn-key login + Coder admin bootstrap shared by all image flavours live in +[`nixos/_images/box-turnkey.nix`](nixos/_images/box-turnkey.nix): autologin to the `coderbox` desktop, and admin `admin@coder.com` / `PleaseChangeMe1234`. Coder comes up at `http://.local:3000` (or the `*.try.coder.app` tunnel URL in `/etc/motd`). Change these before sharing an image by dropping a gitignored @@ -167,7 +175,7 @@ desktop, and admin `admin@coder.com` / `PleaseChangeMe1234`. Coder comes up at The appliance root filesystem is the squashfs + tmpfs overlay from nixpkgs' `iso-image.nix`, so there's no partition to format or mount and **all state is discarded on reboot**. `hosts/_appliance_iso/default.nix` imports -[`nixos/_appliance/live-iso.nix`](nixos/_appliance/live-iso.nix) (which pulls in `box-turnkey.nix`) — +[`nixos/_images/appliance/iso.nix`](nixos/_images/appliance/iso.nix) (which pulls in `base/iso.nix` + `box-turnkey.nix`) — **no** `disko-standard.nix`, `hardware-configuration.nix`, or `facter.json`. The installed-machine `systemd-boot` / EFI-variable settings are forced off; the ISO carries its own GRUB-EFI + isolinux loader (BIOS boot is x86-only, so the @@ -200,11 +208,30 @@ ESP + ext4 root) and **state survives reboots**, exactly like a machine you ran sudo dd if=result/*.img of=/dev/sdX bs=4M status=progress oflag=sync ``` -Both image hosts are completely separate from the disk-install flow above +All image hosts are completely separate from the disk-install flow above (`install.sh`, `nixos-facter`); adding them changes nothing for normal installs. The `_appliance-disk` host shares only the disk *layout* (`disko-standard.nix`) with real installs, never the install process itself. +### Installer ISO (`_installer-iso`) + +The installer is the box as an ISO whose eventual job is to install `coder/box` +onto real hardware. **For now it is identical to the appliance ISO** (full GUI +box + turn-key Coder bootstrap), differing only in image identity (volume ID +`CODER_BOX_INSTALLER`, boot-menu label, and file name +`coder-box-installer-.iso`); the minimal/installer-only environment is a +future change. It builds **only as an ISO** (no qcow2/raw): + +```sh +make installer/iso # native arch +make installer/iso/aarch64-linux # explicit arch +# → out/installer-iso/iso/coder-box-installer-*.iso +``` + +`hosts/_installer-iso/default.nix` imports +[`nixos/_images/installer/iso.nix`](nixos/_images/installer/iso.nix), which — +like the appliance ISO — pulls in `base/iso.nix` + `box-turnkey.nix`. + ## After install The installer auto-creates the admin user, mints a long-lived API token to diff --git a/agents.md b/agents.md index deadcc0..cb463f6 100644 --- a/agents.md +++ b/agents.md @@ -184,17 +184,22 @@ sudo k3s kubectl describe pod -n coder-workspaces k3s-sysbox.nix # k3s + sysbox runtime k3s-podman.nix # k3s + rootless Podman socket screenconnect.nix # ScreenConnect remote access client - _appliance/ # prebuilt-appliance modules (ISO + persistent disk) - box-turnkey.nix # shared turn-key bits for appliances (login + Coder bootstrap) - live-iso.nix # ephemeral appliance ISO module (imported by hosts/_appliance_iso) + _images/ # prebuilt-image modules (appliance + installer) + box-turnkey.nix # shared turn-key Coder box (login + Coder bootstrap); all image hosts + base/hardware.nix # all-hardware (boot on arbitrary hardware) + base/iso.nix # shared ISO mechanics (iso-image.nix, EFI/BIOS/USB bootable, bootloader) + appliance/iso.nix # appliance ISO module (imported by hosts/_appliance_iso) + installer/iso.nix # installer ISO module (imported by hosts/_installer-iso) pkgs/ coder.nix # Coder server package derivation coderd-provider.nix # terraform-provider-coderd derivation hosts/ - _appliance_iso/ # `_appliance_iso` host: ephemeral live "Box" ISO; no disko/facter/hardware-config + _appliance_iso/ # `_appliance_iso` host: ephemeral appliance ISO; no disko/facter/hardware-config # build: make appliance/iso (or appliance/iso/) _appliance-disk/ # `_appliance-disk` host: persistent qcow2/raw disk image (disko image builder) # build: make appliance/qcow2 | make appliance/raw (or .../) + _installer-iso/ # `_installer-iso` host: installer ISO (ISO only; full GUI box for now) + # build: make installer/iso (or installer/iso/) coder-thinkcentre/ # folder name = hostname; default.nix has a hardware-model header comment default.nix # host module: imports facter/legacy + local.nix + thinkcentre-only services hardware-configuration.nix # legacy fallback (used until facter.json exists) diff --git a/flake.nix b/flake.nix index 4c086ba..b5cc7aa 100644 --- a/flake.nix +++ b/flake.nix @@ -41,8 +41,9 @@ # hostname, so `nixos-rebuild switch --flake .` auto-selects the right # config on the running box without needing `.#`. Adding a new host # means just creating ./hosts//default.nix; no flake.nix edit. - # (Underscore-prefixed folders like _appliance_iso are image builds that - # skip the folder-name hostname; see mkHost below.) + # (Underscore-prefixed folders like _appliance_iso, _appliance-disk, and + # _installer-iso are image builds that skip the folder-name hostname; see + # mkHost below.) hostNames = lib.attrNames (lib.filterAttrs (name: type: type == "directory" diff --git a/hosts/_appliance-disk/default.nix b/hosts/_appliance-disk/default.nix index b4d4046..40348b2 100644 --- a/hosts/_appliance-disk/default.nix +++ b/hosts/_appliance-disk/default.nix @@ -20,14 +20,14 @@ # This host is independent of install.sh; it shares the disk LAYOUT with # real installs (disko-standard.nix) but is never itself part of the install # flow. The turn-key login + Coder admin bootstrap (shared with the appliance ISO) -# live in nixos/_appliance/box-turnkey.nix. +# live in nixos/_images/box-turnkey.nix. { lib, pkgs, ... }: { imports = [ - ../../nixos/disko-standard.nix # 1 GB ESP + ext4 root single-disk layout - ../../nixos/_appliance/box-turnkey.nix # shared turn-key config (login + Coder bootstrap) + ../../nixos/disko-standard.nix # 1 GB ESP + ext4 root single-disk layout + ../../nixos/_images/box-turnkey.nix # shared turn-key config (login + Coder bootstrap) ] ++ lib.optional (builtins.pathExists ./local.nix) ./local.nix; # No networking.hostName here on purpose: underscore-prefixed image hosts get diff --git a/hosts/_appliance_iso/default.nix b/hosts/_appliance_iso/default.nix index a9e6a16..28f0f7a 100644 --- a/hosts/_appliance_iso/default.nix +++ b/hosts/_appliance_iso/default.nix @@ -11,7 +11,7 @@ # Unlike the install hosts (coder-thinkcentre, qemu-arm64), this host does NOT # import nixos/disko-standard.nix, hardware-configuration.nix, or facter.json: # the appliance root filesystem is the squashfs + tmpfs overlay provided by -# nixos/_appliance/live-iso.nix. All of the appliance-ISO wiring lives there. +# nixos/_images/appliance/iso.nix. All of the appliance-ISO wiring lives there. # # This host is independent of install.sh and never participates in the # disk-install flow; adding it changes nothing for disko/nixos-install installs. @@ -19,7 +19,7 @@ { lib, ... }: { - imports = [ ../../nixos/_appliance/live-iso.nix ] + imports = [ ../../nixos/_images/appliance/iso.nix ] ++ lib.optional (builtins.pathExists ./local.nix) ./local.nix; # No networking.hostName here on purpose: underscore-prefixed image hosts get diff --git a/hosts/_installer-iso/default.nix b/hosts/_installer-iso/default.nix new file mode 100644 index 0000000..7570ba9 --- /dev/null +++ b/hosts/_installer-iso/default.nix @@ -0,0 +1,29 @@ +# Installer ISO host — boots the Coder box to install it onto real hardware. +# +# Folder name = nixosConfigurations attribute (see flake.nix host +# auto-discovery), so this host is exposed as `nixosConfigurations._installer-iso`. +# It's normally built via the Makefile rather than by attribute: +# +# make installer/iso # → out/installer-iso/iso/coder-box-installer-*.iso +# # equivalently: +# nix build .#nixosConfigurations._installer-iso.config.system.build.isoImage +# +# For now the installer is identical to the appliance ISO (full GUI box + +# turn-key Coder bootstrap) and differs only in image identity; the eventual +# minimal, GUI-less installer environment is deferred. Unlike the appliance, the +# installer ships ONLY as an ISO (no qcow2/raw disk images). All of the +# installer-ISO wiring lives in nixos/_images/installer/iso.nix. +# +# This host is independent of install.sh and never participates in the +# disk-install flow; adding it changes nothing for disko/nixos-install installs. + +{ lib, ... }: + +{ + imports = [ ../../nixos/_images/installer/iso.nix ] + ++ lib.optional (builtins.pathExists ./local.nix) ./local.nix; + + # No networking.hostName here on purpose: underscore-prefixed image hosts get + # no folder-name injection from flake.nix and inherit the central default + # "coder-box" (configuration.nix). Override in local.nix if you need another. +} diff --git a/install.sh b/install.sh index 2d556bf..3de6fdb 100755 --- a/install.sh +++ b/install.sh @@ -31,6 +31,28 @@ set -euo pipefail REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# ── Writable working copy ────────────────────────────────────────────────── +# install.sh writes generated host files into the repo (hosts//...), so +# REPO_DIR must be writable. On the normal live-USB flow it's a writable git +# clone. On the installer/appliance ISO the repo is baked at /etc/nixos-repo, a +# symlink into the read-only Nix store, so writing fails ("Read-only file +# system"). When REPO_DIR isn't writable, copy it to a writable tmpdir +# (tmpfs/RAM on a live ISO) and re-exec from there. The copy is a verbatim +# `cp -a`, so if the baked repo carries a .git the copy keeps it (with its +# origin) and the installed /etc/nixos-repo can still `git pull`. +if [[ ! -w "$REPO_DIR" ]]; then + workdir="$(mktemp -d "${TMPDIR:-/tmp}/coder-box-install.XXXXXX")" + echo "=== Repo at $REPO_DIR is read-only; copying to a writable dir at $workdir/box ===" >&2 + cp -a "$REPO_DIR/." "$workdir/box/" + chmod -R u+w "$workdir/box" + # Signal to the re-exec that we're installing FROM a baked Coder box image + # (read-only repo in the Nix store) rather than a plain live-USB clone. The + # install step uses this to copy the prebuilt closure into /mnt explicitly + # (see "Install" below). + export CODER_BOX_FROM_IMAGE=1 + exec "$workdir/box/install.sh" "$@" +fi + # ── Flag parsing ─────────────────────────────────────────────────────────── HOSTNAME_ARG="" HARDWARE_DESC_ARG="" @@ -47,18 +69,24 @@ ASSUME_YES=0 usage() { sed -n '2,/^set -euo/p' "$0" | sed 's/^# \?//; s/^set -euo.*//' | sed '/^$/N;/^\n$/D'; } +# Require a value for a value-taking flag. Without this, a flag passed as the +# last token with no argument (e.g. `--coder-admin-password`) expands `$2` under +# `set -u` and crashes with "$2: unbound variable" instead of a clear message. +need_value() { + [[ $# -ge 2 ]] || { echo "flag $1 requires a value" >&2; usage >&2; exit 2; } +} while [[ $# -gt 0 ]]; do case "$1" in - --hostname) HOSTNAME_ARG="$2"; shift 2 ;; - --hardware-desc) HARDWARE_DESC_ARG="$2"; shift 2 ;; - --disk) DISK_ARG="$2"; shift 2 ;; - --coder-admin-email) ADMIN_EMAIL_ARG="$2"; shift 2 ;; - --coder-admin-password) ADMIN_PASSWORD_ARG="$2"; shift 2 ;; - --coder-admin-password-file) ADMIN_PASSWORD_FILE_ARG="$2"; shift 2 ;; - --nixos-username) NIXOS_USERNAME_ARG="$2"; shift 2 ;; - --nixos-password) NIXOS_PASSWORD_ARG="$2"; shift 2 ;; - --nixos-password-file) NIXOS_PASSWORD_FILE_ARG="$2"; shift 2 ;; - --lan-ip) LAN_IP_ARG="$2"; shift 2 ;; + --hostname) need_value "$@"; HOSTNAME_ARG="$2"; shift 2 ;; + --hardware-desc) need_value "$@"; HARDWARE_DESC_ARG="$2"; shift 2 ;; + --disk) need_value "$@"; DISK_ARG="$2"; shift 2 ;; + --coder-admin-email) need_value "$@"; ADMIN_EMAIL_ARG="$2"; shift 2 ;; + --coder-admin-password) need_value "$@"; ADMIN_PASSWORD_ARG="$2"; shift 2 ;; + --coder-admin-password-file) need_value "$@"; ADMIN_PASSWORD_FILE_ARG="$2"; shift 2 ;; + --nixos-username) need_value "$@"; NIXOS_USERNAME_ARG="$2"; shift 2 ;; + --nixos-password) need_value "$@"; NIXOS_PASSWORD_ARG="$2"; shift 2 ;; + --nixos-password-file) need_value "$@"; NIXOS_PASSWORD_FILE_ARG="$2"; shift 2 ;; + --lan-ip) need_value "$@"; LAN_IP_ARG="$2"; shift 2 ;; --no-reboot) NO_REBOOT=1; shift ;; --yes|-y) ASSUME_YES=1; shift ;; --help|-h) usage; exit 0 ;; @@ -171,12 +199,25 @@ sed_replacement_escape() { list_disks() { # Whole-block-devices, non-removable, non-loop, non-rom. MODEL is last so an # empty model (e.g. virtio /dev/vda) can't shift the TYPE/RM columns. + # Skip zram (compressed RAM swap, /dev/zramN) — it reports TYPE=disk RM=0 so + # it would otherwise show up as an install target, which is never what we want + # (installing onto RAM swap). Also skip device-mapper / md / loop just in case. lsblk -d -p -n -b -o NAME,SIZE,RM,TYPE,MODEL \ - | awk '$4=="disk" && $3=="0" { size_h=$2; cmd="numfmt --to=iec --suffix=B "$2; cmd|getline size_h; close(cmd); model=""; for(i=5;i<=NF;i++) model=model (model==""?"":" ") $i; print $1"\t"size_h"\t"model }' + | awk '$4=="disk" && $3=="0" && $1 !~ /\/(zram|dm-|md|loop)[0-9]+$/ { size_h=$2; cmd="numfmt --to=iec --suffix=B "$2; cmd|getline size_h; close(cmd); model=""; for(i=5;i<=NF;i++) model=model (model==""?"":" ") $i; print $1"\t"size_h"\t"model }' } # ── Gather inputs ────────────────────────────────────────────────────────── +# Resolve the build/commit revision for display: prefer git (the normal +# live-USB clone, or a fork checkout), else the baked /etc/coder-box-rev that +# the box image writes (its /etc/nixos-repo has no .git), else "unknown". +box_revision() { + local rev + rev="$(git -C "$REPO_DIR" rev-parse HEAD 2>/dev/null)" && { echo "$rev"; return; } + rev="$(cat /etc/coder-box-rev 2>/dev/null)" && [[ -n "$rev" ]] && { echo "$rev"; return; } + echo "unknown" +} echo "=== Coder NixOS installer ===" +echo " revision: $(box_revision)" echo # Defaults used when the corresponding flag is omitted. @@ -357,11 +398,16 @@ if [[ ! -f "$HOST_DIR/facter.json" ]]; then echo " wrote hosts/$HOSTNAME_ARG/facter.json" fi -# local.nix is gitignored, so force-add as intent-to-add; the others normal. -git -C "$REPO_DIR" add --intent-to-add -f \ - "hosts/$HOSTNAME_ARG/default.nix" \ - "hosts/$HOSTNAME_ARG/facter.json" \ - "hosts/$HOSTNAME_ARG/local.nix" >/dev/null +# A git path flake ignores untracked files, so the freshly written host files +# must be intent-to-added for the flake to see them (local.nix is gitignored, so +# force-add it). Only meaningful when REPO_DIR is a git repo; the ISO writable +# copy may have no .git (a non-git path flake already sees every file), so skip. +if git -C "$REPO_DIR" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git -C "$REPO_DIR" add --intent-to-add -f \ + "hosts/$HOSTNAME_ARG/default.nix" \ + "hosts/$HOSTNAME_ARG/facter.json" \ + "hosts/$HOSTNAME_ARG/local.nix" >/dev/null +fi # ── Validate ─────────────────────────────────────────────────────────────── echo " validating flake ..." @@ -403,13 +449,59 @@ mkdir -p /mnt/etc/nixos ln -sf /etc/nixos-repo/flake.nix /mnt/etc/nixos/flake.nix # ── Install ──────────────────────────────────────────────────────────────── -echo "=== Running nixos-install ===" -echo " (closure builds into /mnt/nix/store; no tmpfs OOM risk)" -nixos-install \ - --flake "/mnt/etc/nixos-repo#${HOSTNAME_ARG}" \ - --no-channel-copy \ - --no-root-passwd \ - --option download-buffer-size 268435456 +# Two cases: +# +# (A) Installing FROM a Coder box image (installer/appliance ISO; +# CODER_BOX_FROM_IMAGE=1). The live /nix/store already contains almost the +# entire closure for the target system — but in the read-only squashfs lower +# layer of the overlay. nixos-install builds the flake with +# `nix build --store /mnt --extra-substituters `, i.e. it +# *substitutes* paths into /mnt from the host store; squashfs paths lack the +# signatures/narinfo substitution needs, so the copy silently yields +# nothing. /mnt is left empty (no bash; the `system` profile points at the +# wrong path) and the chroot `activate` fails: "No such file or directory". +# (NOTE: the target host is `coder-nixos`, a *different* system than the +# image's own host, so its toplevel isn't pre-realised — it must be built. +# The build is cheap: every heavy dependency (KDE, Coder, k3s, …) is reused +# from the squashfs; only the few host-specific derivations are new.) +# +# So: build the toplevel, copy its full closure into /mnt with +# `nix copy --no-check-sigs` (bypassing the signature/substituter machinery +# that was the actual failure), then `nixos-install --system ` just +# activates it. This mirrors the working manual workaround ("copy the repo +# somewhere writable and run install.sh"). +# +# (B) Plain live-USB clone (stock NixOS ISO). The closure is NOT present, so +# building it in the host store first would balloon tmpfs/RAM. Keep the +# original `nixos-install --flake` which builds/downloads straight into +# /mnt. +if [[ "${CODER_BOX_FROM_IMAGE:-0}" == "1" ]]; then + echo "=== Building system closure (reusing the baked store) ===" + SYSTEM_TOPLEVEL=$(nix --extra-experimental-features 'nix-command flakes' \ + build --no-link --print-out-paths \ + --option download-buffer-size 268435456 \ + "/mnt/etc/nixos-repo#nixosConfigurations.${HOSTNAME_ARG}.config.system.build.toplevel") + [[ -n "$SYSTEM_TOPLEVEL" ]] || { echo "failed to build system closure" >&2; exit 1; } + + echo "=== Copying system closure into /mnt ===" + nix --extra-experimental-features 'nix-command flakes' \ + copy --no-check-sigs --to "local?root=/mnt" "$SYSTEM_TOPLEVEL" + + echo "=== Running nixos-install (from prebuilt system) ===" + nixos-install \ + --system "$SYSTEM_TOPLEVEL" \ + --no-channel-copy \ + --no-root-passwd \ + --option download-buffer-size 268435456 +else + echo "=== Running nixos-install ===" + echo " (closure builds into /mnt/nix/store; no tmpfs OOM risk)" + nixos-install \ + --flake "/mnt/etc/nixos-repo#${HOSTNAME_ARG}" \ + --no-channel-copy \ + --no-root-passwd \ + --option download-buffer-size 268435456 +fi echo echo "✓ Installation complete." diff --git a/nixos/_appliance/live-iso.nix b/nixos/_appliance/live-iso.nix deleted file mode 100644 index df63ac9..0000000 --- a/nixos/_appliance/live-iso.nix +++ /dev/null @@ -1,66 +0,0 @@ -# Appliance ISO module — "it's just The Box™", not an installer. -# -# Turns the shared Coder box configuration into a bootable *ephemeral* appliance -# ISO that runs entirely from the USB/CD + RAM, with no disk install. Booting it -# gives the same system the on-disk install produces (KDE, Coder server, k3s, -# Podman, the bundled templates, all started automatically) — but the root -# filesystem is a squashfs + tmpfs overlay, so all state is discarded on -# reboot. For a *persistent* appliance (state survives reboots) build the -# _appliance-disk host instead (qcow2 / raw); see the Makefile / README. -# -# Build (hosts/_appliance_iso => nixosConfigurations._appliance_iso, see flake.nix): -# -# make appliance/iso -# # or: nix build .#nixosConfigurations._appliance_iso.config.system.build.isoImage -# # → out/appliance-iso/iso/coder-box-appliance-*.iso (flash with `dd`, Ventoy, etc.) -# -# This module is imported only by hosts/_appliance_iso/default.nix and is -# independent of the regular disk-install flow (install.sh, disko, -# nixos-facter). It imports NO disko / hardware-configuration.nix / facter.json: -# the appliance root is the squashfs + tmpfs overlay that nixpkgs' iso-image.nix -# sets up. -# -# The turn-key login + Coder admin bootstrap (shared with the _appliance-disk -# image) live in nixos/_appliance/box-turnkey.nix. - -{ config, lib, pkgs, modulesPath, ... }: - -{ - imports = [ - # Core ISO builder: squashfs nix store, tmpfs overlay root, kernel/initrd, - # and the EFI + BIOS ISO bootloader. Provides `system.build.isoImage` and - # the `isoImage.*` options used below. - (modulesPath + "/installer/cd-dvd/iso-image.nix") - # Shared turn-key config (all-hardware, baked /etc/nixos-repo, autologin, - # Coder admin bootstrap). - ./box-turnkey.nix - ]; - - # ── ISO image settings ────────────────────────────────────────────────────── - isoImage.makeEfiBootable = true; # boot on UEFI machines - # Legacy BIOS boot uses syslinux, which is x86-only. Enable it just for x86 - # so the same module also evaluates/builds for an aarch64 appliance ISO (which - # boots via EFI only). isx86 covers both i686 and x86_64. - isoImage.makeBiosBootable = pkgs.stdenv.hostPlatform.isx86; - isoImage.makeUsbBootable = true; # `dd` straight to a USB stick and boot - isoImage.volumeID = "CODER_BOX_LIVE"; - # Boot-menu label (both the BIOS/isolinux and EFI/grub entries). The label is - # " "; the default append is - # " Installer", which is misleading here since this is the live appliance, not - # the installer. Append " - Coder Box Appliance" -> "NixOS - Coder - # Box Appliance". Leading space is required (it's concatenated directly). - isoImage.appendToMenuLabel = " - Coder Box Appliance"; - # ISO file name. iso-image.nix derives isoName from image.baseName as - # ".iso", and defaults baseName to "nixos--". We - # override baseName (mkForce, to win over that default) but keep the arch - # suffix so the file is e.g. coder-box-appliance-aarch64-linux.iso — the arch - # is visible in the name and x86_64/aarch64 ISOs don't collide in ./out. - image.baseName = lib.mkForce "coder-box-appliance-${pkgs.stdenv.hostPlatform.system}"; - - # ── Boot loader: let iso-image.nix own it ──────────────────────────────────── - # configuration.nix sets these for installed UEFI machines; force them off so - # they don't conflict with the image's own bootloader or try to touch the - # host's EFI variables when the live system activates. - boot.loader.systemd-boot.enable = lib.mkForce false; - boot.loader.efi.canTouchEfiVariables = lib.mkForce false; -} diff --git a/nixos/_images/appliance/iso.nix b/nixos/_images/appliance/iso.nix new file mode 100644 index 0000000..553042c --- /dev/null +++ b/nixos/_images/appliance/iso.nix @@ -0,0 +1,44 @@ +# Appliance ISO module — "it's just The Box™", not an installer. +# +# Turns the shared Coder box configuration into a bootable *ephemeral* appliance +# ISO that runs entirely from the USB/CD + RAM, with no disk install. Booting it +# gives the same system the on-disk install produces (KDE, Coder server, k3s, +# Podman, the bundled templates, all started automatically) — but the root +# filesystem is a squashfs + tmpfs overlay, so all state is discarded on +# reboot. For a *persistent* appliance (state survives reboots) build the +# _appliance-disk host instead (qcow2 / raw); see the Makefile / README. +# +# Build (hosts/_appliance_iso => nixosConfigurations._appliance_iso, see flake.nix): +# +# make appliance/iso +# # or: nix build .#nixosConfigurations._appliance_iso.config.system.build.isoImage +# # → out/appliance-iso/iso/coder-box-appliance-*.iso (flash with `dd`, Ventoy, etc.) +# +# Composition: the ISO mechanics (iso-image.nix, EFI/BIOS/USB bootable, +# bootloader overrides, all-hardware) live in ../base/iso.nix; the turn-key +# Coder box (baked /etc/nixos-repo, nixpkgs registry, coderbox autologin, Coder +# admin bootstrap) lives in ../box-turnkey.nix. This module only sets the +# appliance's image identity. + +{ config, lib, pkgs, ... }: + +{ + imports = [ + ../base/iso.nix # shared ISO mechanics + ../box-turnkey.nix # shared turn-key Coder box (login + Coder bootstrap) + ]; + + # ── Image identity ─────────────────────────────────────────────────────────── + isoImage.volumeID = "CODER_BOX_APPLIANCE"; + # Boot-menu label (both the BIOS/isolinux and EFI/grub entries). The label is + # " "; the default append is + # " Installer", which is misleading here since this is the appliance, not the + # installer. Leading space is required (it's concatenated directly). + isoImage.appendToMenuLabel = " - Coder Box Appliance"; + # ISO file name. iso-image.nix derives isoName from image.baseName as + # ".iso", and defaults baseName to "nixos--". We + # override baseName (mkForce, to win over that default) but keep the arch + # suffix so the file is e.g. coder-box-appliance-aarch64-linux.iso — the arch + # is visible in the name and x86_64/aarch64 ISOs don't collide in ./out. + image.baseName = lib.mkForce "coder-box-appliance-${pkgs.stdenv.hostPlatform.system}"; +} diff --git a/nixos/_images/base/hardware.nix b/nixos/_images/base/hardware.nix new file mode 100644 index 0000000..f4126f9 --- /dev/null +++ b/nixos/_images/base/hardware.nix @@ -0,0 +1,9 @@ +# Broad driver/firmware set so an image boots on arbitrary real hardware or +# virtual machines. Single source of `all-hardware.nix` for the _images tree: +# imported by both base/iso.nix and ../box-turnkey.nix. This replaces the +# per-host facter.json / hardware-configuration.nix that installed hosts rely +# on (image hosts ship neither). +{ modulesPath, ... }: +{ + imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; +} diff --git a/nixos/_images/base/iso.nix b/nixos/_images/base/iso.nix new file mode 100644 index 0000000..e0f31ac --- /dev/null +++ b/nixos/_images/base/iso.nix @@ -0,0 +1,37 @@ +# Shared ISO mechanics for every Coder box image that ships as an ISO +# (appliance ISO, installer ISO). This is a _base primitive: it wires up the +# nixpkgs ISO builder and the boot-loader overrides, but carries NO image +# identity (volumeID / menu label / file name) — each image module under +# _images/appliance or _images/installer sets those. +# +# Provides `config.system.build.isoImage` and the `isoImage.*` options. +{ config, lib, pkgs, modulesPath, ... }: +{ + imports = [ + # Core ISO builder: squashfs nix store, tmpfs overlay root, kernel/initrd, + # and the EFI + BIOS ISO bootloader. + (modulesPath + "/installer/cd-dvd/iso-image.nix") + # Broad driver/firmware set (see hardware.nix). + ./hardware.nix + ]; + + isoImage.makeEfiBootable = true; # boot on UEFI machines + # Legacy BIOS boot uses syslinux, which is x86-only. Enable it just for x86 so + # the same module also evaluates/builds for an aarch64 ISO (which boots via + # EFI only). isx86 covers both i686 and x86_64. + isoImage.makeBiosBootable = pkgs.stdenv.hostPlatform.isx86; + isoImage.makeUsbBootable = true; # `dd` straight to a USB stick and boot + + # Boot loader: let iso-image.nix own it. configuration.nix sets these for + # installed UEFI machines; force them off so they don't conflict with the + # image's own bootloader or try to touch the host's EFI variables when the + # live system activates. + boot.loader.systemd-boot.enable = lib.mkForce false; + boot.loader.efi.canTouchEfiVariables = lib.mkForce false; + + # dmidecode: read SMBIOS/DMI (board model, BIOS, serial). install.sh's + # hardware-description auto-detection uses it, and it's handy for inspecting + # the target machine from the live ISO. Ship it in the base layer so every + # ISO flavour (installer + appliance) has it. + environment.systemPackages = [ pkgs.dmidecode ]; +} diff --git a/nixos/_appliance/box-turnkey.nix b/nixos/_images/box-turnkey.nix similarity index 68% rename from nixos/_appliance/box-turnkey.nix rename to nixos/_images/box-turnkey.nix index 586174b..b4ad6f1 100644 --- a/nixos/_appliance/box-turnkey.nix +++ b/nixos/_images/box-turnkey.nix @@ -1,9 +1,10 @@ # Shared "turn-key" Box™ config — the bits that make an image boot straight # into a fully-configured, ready-to-use Coder box with no install step. # -# Imported by both appliance flavours: -# - nixos/_appliance/live-iso.nix (ephemeral appliance ISO: hosts/_appliance_iso) -# - hosts/_appliance-disk/ (persistent disk image: qcow2 / raw) +# Shared by every Coder box image flavour: +# - nixos/_images/appliance/iso.nix (ephemeral appliance ISO: hosts/_appliance_iso) +# - nixos/_images/installer/iso.nix (installer ISO: hosts/_installer-iso) +# - hosts/_appliance-disk/ (persistent disk image: qcow2 / raw) # # On real installs these settings come from install.sh + the gitignored # hosts//local.nix it generates. The image flavours have no install step, @@ -11,17 +12,36 @@ # defaults to). Change them before handing an image to anyone untrusted, or # override per-image via hosts//local.nix. -{ config, lib, pkgs, modulesPath, self, inputs, ... }: +{ config, lib, pkgs, self, inputs, ... }: { imports = [ # Broad driver/firmware set so the image boots on arbitrary hardware / - # virtual machines. This replaces the per-host facter.json / - # hardware-configuration.nix that installed hosts rely on (image hosts - # ship neither). - (modulesPath + "/profiles/all-hardware.nix") + # virtual machines (single source for the _images tree). Replaces the + # per-host facter.json / hardware-configuration.nix that installed hosts + # rely on (image hosts ship neither). The ISO flavours also pull this in + # via base/iso.nix; importing the same module twice is a harmless no-op + # (NixOS dedups identical module paths), and the _appliance-disk host — + # which imports box-turnkey but NOT iso.nix — needs it from here. + ./base/hardware.nix ]; + # Build revision baked into the image (used by the installer's boot-menu label + # and /etc/coder-box-rev). Default works for `.#` (git+file) builds where + # `self` carries git metadata; the Makefile builds through a *path* flakeref + # (getFlake (toString ./.)) which has NO git metadata, so it overrides this + # with `coderBox.rev = ""` (see Makefile box_build). Defined here (in + # the shared module) so it exists for every image host the Makefile builds. + # (Because this module declares `options`, all its config must live under the + # `config = { … }` block below.) + options.coderBox.rev = lib.mkOption { + type = lib.types.str; + default = self.rev or self.dirtyRev or "unknown"; + description = "Git revision this Coder box image was built from."; + }; + + config = { + # ── Bake the repo into the image at /etc/nixos-repo ────────────────────────── # The on-disk installer copies the working tree to /etc/nixos-repo; the Coder # bootstrap units (coder-init-admin.service, the coder-template-sync @@ -82,4 +102,6 @@ CODER_ADMIN_USERNAME = "admin"; CODER_ADMIN_PASSWORD = "PleaseChangeMe1234"; }; + + }; # end config } diff --git a/nixos/_images/installer/iso.nix b/nixos/_images/installer/iso.nix new file mode 100644 index 0000000..46c74e1 --- /dev/null +++ b/nixos/_images/installer/iso.nix @@ -0,0 +1,136 @@ +# Installer ISO module — boots the Coder box to install it onto real hardware. +# +# For now this is intentionally identical to the appliance ISO (full GUI box + +# turn-key Coder bootstrap), differing in image identity, an auto-run installer +# console, and having the Coder runtime services switched off (the installer +# doesn't need a live Coder stack). The eventual minimal, GUI-less installer is +# deferred. +# +# Build (hosts/_installer-iso => nixosConfigurations._installer-iso, see flake.nix): +# +# make installer/iso +# # or: nix build .#nixosConfigurations._installer-iso.config.system.build.isoImage +# # → out/installer-iso/iso/coder-box-installer-*.iso (flash with `dd`, Ventoy, etc.) +# +# Composition mirrors the appliance ISO: ../base/iso.nix (ISO mechanics) + +# ../box-turnkey.nix (turn-key Coder box). Unlike the appliance, the installer +# is built ONLY as an ISO (no qcow2/raw disk images). + +{ config, lib, pkgs, self, ... }: + +let + boxRev = config.coderBox.rev; + # Short form for the boot-menu label (full 40-char hashes are unwieldy there). + boxRevShort = if boxRev == "unknown" then "unknown" else builtins.substring 0 12 boxRev; + + # Launcher run inside the preopened Konsole: cd into the baked repo, run the + # installer as root (passwordless sudo is configured in configuration.nix), + # and — whatever happens — drop the user into an interactive bash shell so a + # failed install leaves them at a prompt to inspect/retry instead of a dead + # terminal. (On success install.sh reboots, so the shell is only reached on + # failure or --no-reboot.) + installerLauncher = pkgs.writeShellScript "coder-box-installer-launch" '' + cd /etc/nixos-repo 2>/dev/null || cd / + echo "=== Coder Box installer ===" + echo "Running: sudo ./install.sh $*" + echo + if sudo ./install.sh "$@"; then + echo + echo "=== install.sh finished ===" + else + rc=$? + echo + echo "=== install.sh FAILED (exit $rc) — dropping you into a shell ===" + echo " You are in /etc/nixos-repo. Re-run with: sudo ./install.sh" + fi + echo + # Interactive login-ish shell so the user can debug/retry. exec so closing + # the shell closes Konsole. + exec ${pkgs.bashInteractive}/bin/bash -i + ''; +in +{ + imports = [ + ../base/iso.nix # shared ISO mechanics + ../box-turnkey.nix # shared turn-key Coder box (login + Coder bootstrap) + ]; + + # config.coderBox.rev is defined in ../box-turnkey.nix (shared by all image + # hosts so the Makefile can inject the rev for every target). It defaults to + # self.rev/dirtyRev for `.#` builds and is overridden by the Makefile. + config = { + # ── Image identity ───────────────────────────────────────────────────────── + isoImage.volumeID = "CODER_BOX_INSTALLER"; + # Boot-menu label (BIOS/isolinux + EFI/grub). See ../appliance/iso.nix for + # the format; leading space is required. Include the short build revision so + # the boot menu shows exactly which image you're booting. + isoImage.appendToMenuLabel = " - Coder Box Installer (${boxRevShort})"; + + # Record the full build revision for install.sh to print (the baked repo + # under /etc/nixos-repo has no .git, so the script can't get it from git). + environment.etc."coder-box-rev".text = boxRev + "\n"; + + # ISO file name, with arch suffix (e.g. coder-box-installer-x86_64-linux.iso). + # See ../appliance/iso.nix for why this is mkForce + arch-suffixed. + image.baseName = lib.mkForce "coder-box-installer-${pkgs.stdenv.hostPlatform.system}"; + + # ── Auto-launch a full-screen Konsole that runs the installer ────────────── + # box-turnkey.nix autologins straight into the Plasma (X11) desktop. For the + # installer we want the install to start on its own: a system-wide XDG + # autostart entry opens Konsole full-screen on session start and runs the + # installer launcher (`konsole -e `), which `sudo ./install.sh`s + # and drops to an interactive bash shell if it fails. + environment.systemPackages = [ pkgs.kdePackages.konsole ]; + environment.etc."xdg/autostart/coder-box-installer-konsole.desktop".text = '' + [Desktop Entry] + Type=Application + Name=Coder Box Installer Console + Comment=Run the coder/box installer in a full-screen terminal + Exec=${pkgs.kdePackages.konsole}/bin/konsole --fullscreen --workdir /etc/nixos-repo -e ${installerLauncher} + Terminal=false + X-GNOME-Autostart-enabled=true + OnlyShowIn=KDE; + ''; + + # ── Never prompt for a password to get in ────────────────────────────────── + # Login is already passwordless (box-turnkey coderbox autologin + passwordless + # sudo). Disable KDE's screen locker (idle auto-lock / lock-on-resume) so the + # installer is never locked. (The appliance keeps the default locker.) + environment.etc."xdg/kscreenlockerrc".text = '' + [Daemon] + Autolock=false + LockOnResume=false + ''; + + # ── Installer ergonomics ─────────────────────────────────────────────────── + # `sudo ./install.sh` from the baked /etc/nixos-repo works because the script + # detects its repo dir is read-only (a symlink into the read-only Nix store) + # and copies it to a writable tmpdir (tmpfs/RAM here) to re-exec from. The + # copy keeps the baked .git (if any) so the installed /etc/nixos-repo can + # `git pull`. Mirror nixpkgs' installation-device.nix low-memory tweak so the + # kernel's overcommit heuristics don't spuriously block forks during install. + boot.kernel.sysctl."vm.overcommit_memory" = "1"; + + # ── Don't run the Coder box services in the installer ────────────────────── + # The installer's only job is to install coder/box onto a disk; it inherits + # the full box config but the running Coder server, k3s, PostgreSQL, Podman, + # the bootstrap/redirect/reaper units, and template-sync are dead weight here + # (slow startup, wasted RAM/CPU during install). Disable them — the INSTALLED + # system still gets everything; this only affects the live installer. + services.coder-nixos.k3s-sysbox.enable = lib.mkForce false; + services.postgresql.enable = lib.mkForce false; + virtualisation.podman.enable = lib.mkForce false; + + systemd.services.coder.enable = false; + systemd.services.coder-init-admin.enable = false; + systemd.services.coder-redirect.enable = false; + systemd.services.coder-logstream-kube.enable = false; + systemd.services.coder-workspace-reaper.enable = false; + systemd.timers.coder-workspace-reaper.enable = false; + systemd.services.coder-sync-ssh-keys.enable = false; + + # Template-sync activation script — pointless in the live installer (no + # running Coder, empty session token). + system.activationScripts.coder-template-sync = lib.mkForce ""; + }; +}