Skip to content

Headless drive e2e + iOS/Android e2e CI jobs + faster test setup - #12

Merged
tiensonqin merged 28 commits into
mainfrom
devin/1790251566-headless-drive-e2e
Sep 24, 2026
Merged

tiensonqin merged 28 commits into
mainfrom
devin/1790251566-headless-drive-e2e

Conversation

@tiensonqin

@tiensonqin tiensonqin commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Test-time work in four parts, plus the fixes needed to un-break CI (main was failing before this PR):

1. Headless e2e via drive, running in the existing CI gate. A new drive alcotest suite mounts the real chat app in-process — Drive.Session + Lui_app.create with a drain host stub that answers pending_effects with canned projections (mirroring the native bridge's Dequeue/ApplyCoreSnapshot/ResolveEffect sequences) — then replays .drive scenario scripts checked into shared/test/logseq_chat/drive/. Nine scenarios cover cold-start → create-graph, composer capture, settings (language radio, appearance theme, tabs sheet), encrypted-graph unlock, native search, graph switch + context-menu delete, journal rendering → tap-to-edit → editor toolbar, and outliner text-change → return commit. The suite sits inside @shared/native/runtest, so ocaml-core CI runs it — no simulator, ~3s total. The host stub seeds two journal rows and answers TapOutlinerBlockEffect/ChangeOutlinerTextEffect/ReturnOutlinerEditorEffect with projection_outliner_editing snapshots and updated rows. New drive pin — logseq/drive#1 (scenario change command + empty-string token fix).

  • Selector gotcha worth knowing: text:"Confirm" substring-matches any string prop — button.graph-add.confirm's a11y id matches too — so iOS dialog buttons (which have no a11y id) need kind:button&prop:text=Confirm (exact prop equality). Similarly text:"Theme" hits the label text node, not the select — use kind:select.

2. iOS/Android e2e in CI (new .github/workflows/e2e.yml):

  • ios-e2e on macos-15: Maestro + opam/duniverse setup (same cache steps as ocaml-core), Xcode 26 selected for lui's swift-tools 6.2, build-mobile-ios-simulator.sh, boot a freshly-created simulator, run the smoke module.
  • android-e2e on ubuntu-latest + KVM: Flutter/Gradle/Maestro setup, reactivecircus/android-emulator-runner boots an x86_64 emulator, runs the smoke module (signed-out + local-graph-setup + capture). LOGSEQ_CHAT_ANDROID_ABI=x86_64 cross-builds liblogseq_chat_core.so for the emulator arch via Gradle → build-android-native.sh.
  • db-sync server runs in CI (./.github/actions/db-sync-server): checks out logseq/logseq@feature/native-mobile-app, installs JDK21 + Clojure CLI + pnpm, clojure -M:cljs release db-sync-node, starts DB_SYNC_PORT=8787 in the background (~70s warm; ~/.m2/~/.gitlibs/pnpm store/node_modules cached). Both jobs point the app at http://127.0.0.1:8787 — iOS reaches it via simulator host loopback, Android via the script's adb reverse tcp:8787 (now gated on LOGSEQ_CHAT_E2E_BASE_URL). Smoke flows sign in through the in-app Cognito form ("Use without an account" → e2etest/[REDACTED SECRET], overridable via LOGSEQ_CHAT_E2E_USERNAME/_PASSWORD) → real JWT → real sync graph creation end-to-end.
  • Triggers: every PR + push to main (smoke only), nightly cron runs the full all suites, workflow_dispatch takes module inputs. OCaml cross toolchains (_build/apple-toolchains, _build/android-toolchain, _build/android-*) are cached — warm runs skip the ~10min OCaml-from-source bootstrap. These jobs are inherently minutes-scale (toolchain + build + simulator boot); the ocaml-core gate remains the fast (~1min warm) check.
  • Script fixes needed for CI: build-mobile-ocaml.sh/both e2e scripts resolved dune via opam exec --switch=5.5.0, but CI's setup-ocaml local switch is _opam (unnamed) — now prefer $DUNE/command -v dune. test-android-e2e.sh cd'd to repo_root/Flutter, which only worked on case-insensitive macOS — fixed to flutter/ (runner test updated). Data-wipe per flow now recreates <container>/tmp — the OCaml core writes its initial snapshot there and crashed on every flow after the first.

3. Faster per-flow iOS e2e setup (scripts/test-ios-e2e{,-suite}.sh):

  • Wipe the app's data container + defaults delete instead of simctl uninstall+install per flow — the binary never changes, so reinstalling was pure overhead. LOGSEQ_CHAT_IOS_E2E_REINSTALL=1 restores the old path.
  • One LOGSEQ_CHAT_E2E_RUN_ID + shared LOGSEQ_CHAT_E2E_SEED_CACHE: the first flow per fixture mode runs the Maestro graph-setup + checkpoint wait + logseq_chat_e2e_seed.exe, caches the seeded Documents/graphs; later flows copy it into a fresh container. ~13 outliner-fixture flows → 1 UI setup.

4. CI fixes for the #main-pin regression from #11 (main's ocaml-core was red):

  • ci.yml: the private-pin duniverse clone step only matched #[0-9a-f]{40}; #main pins were never vendored → persistent_sorted_set_ocaml resolved from _opam and the workspace → dune library-name conflict. Now matches any #ref with a FETCH_HEAD fallback.
  • ci.yml: setup-ocaml is bypassed when the ~/.opam root + _opam switch caches both hit (it ran an unconditional ~40s opam update); duniverse cache key now includes the workflow files.
  • storage_codec: datascript-ocaml@main added optional storage_{eavt,aevt,avet}_metadata fields to storage_root; decode/encode them so roots round-trip.

Test plan

  • dune build @shared/native/runtest — 716 tests pass locally, including all 9 drive scenarios
  • iOS smoke (build + 4/4 flows incl. Cognito sign-in → sync-graph creation against a locally-built db-sync server) verified end-to-end on a macOS session
  • Android toolchain proven through APK build + app launch on CI; the db-sync-in-CI path is new — first real run may need iteration; LOGSEQ_CHAT_IOS_E2E_REINSTALL=1 restores the old reset path

Link to Devin session: https://app.devin.ai/sessions/9a91e201984a4d4f96ce3e0f86ac6668
Open in Devin Desktop: https://app.devin.ai/desktop/session/9a91e201984a4d4f96ce3e0f86ac6668?variant=devin
Requested by: @tiensonqin

Headless e2e: a 'drive' alcotest suite mounts the chat app in-process
(Drive.Session + a host stub that resolves pending effects with canned
projections) and runs .drive scenario scripts under
shared/test/logseq_chat/drive/. Three scenarios cover graph creation,
composer capture, and settings language/tabs. They ride the existing
@shared/native/runtest gate so CI runs them — no simulator, microseconds
per poll. Drive pin bumped to 2ac231e for the 'change' command and
empty-string tokenizer fix (logseq/drive#1).

iOS e2e runtime: test-ios-e2e.sh now wipes the app data container
instead of uninstalling+reinstalling the identical binary per flow
(LOGSEQ_CHAT_IOS_E2E_REINSTALL=1 restores the old path), and the suite
exports one run id plus a per-fixture-mode seed cache so the graph-setup
Maestro flow + sqlite checkpoint wait runs once per mode instead of once
per flow.
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

PR #11 moved datascript-ocaml/melange pins to #main, but the duniverse
clone step only matched #[0-9a-f]{40} — those pins were never vendored,
so _opam's datascript_ocaml resolved persistent_sorted_set_ocaml from
_opam while the workspace used the vendored copy: library-name conflict
in dune build @shared/native/runtest (also failing main).

Match any #ref and fall back to FETCH_HEAD when the ref isn't a local
checkout.
The vendored-dirs cache was restored across changes to the pin-clone
step itself (same opam files => same key), so a duniverse built with
the old sha-only clone logic shadowed the new regex. Hash ci.yml too.
datascript-ocaml@main added optional storage_eavt/aevt/avet_metadata
fields to storage_root (datascript_types). Decode the eavt-metadata /
aevt-metadata / avet-metadata transit keys into them, and emit them
when encoding a root without an explicit index_metadata param so
encode/decode stays a round-trip.
…merged SHA

- Cache ~/.opam (opam root: config, repo index, download cache) under a
  compiler-only key. When it and the _opam switch cache both hit, skip
  ocaml/setup-ocaml entirely — its unconditional 'opam update' costs
  ~40s of the ~64s step — and just install the opam 2.6.0 binary.
  Saves ~60s of fixed overhead per CI run.
- Point the drive pin at the rebase-merged commits on drive main
  (7f407b0, tree-identical to the previous branch SHA).
e2e.yml adds ios-e2e (macos-15, smoke module by default) and android-e2e
(ubuntu + KVM emulator, capture module), triggered on PRs, main pushes,
nightly (full suite), and workflow_dispatch with module inputs. Both jobs
reuse the opam root/switch and duniverse caches from ci.yml; the OCaml
cross toolchains and the Flutter/Gradle caches are keyed separately.

Scripts: resolve dune via $DUNE/PATH first so CI's local _opam switch
(unnamed, not 5.5.0) works; fix repo_root/Flutter -> flutter which broke
on case-sensitive Linux.

New drive scenarios: encrypted-graph-unlock, search-nodes,
graph-switch-delete, journal-outliner. The host stub now seeds two
journal rows and answers TapOutlinerBlockEffect with an editing
projection. Selector gotcha fixed: text:"Confirm" substring-matches
button.graph-add.confirm's a11y id; use prop:text=Confirm for exact.
@devin-ai-integration devin-ai-integration Bot changed the title Headless drive e2e in CI + faster per-flow iOS e2e setup Headless drive e2e + iOS/Android e2e CI jobs + faster test setup Sep 24, 2026
…unbuildable on macOS arm64); derive dune-workspace.mobile switch from env
Apple marks libffi unavailable in iOS SDK headers and the Android NDK
ships none; ctypes-foreign's discover.exe needs real libffi headers via
pkg-config and the final app/.so link needs a real -lffi. Build a static
libffi 3.4.8 per target (cached inside the toolchain dirs) and pass its
pkgconfig dir + archive to the builds.
- build-mobile-libffi: send build output to stderr so $() captures only the prefix
- build-mobile-ocaml: resolve opam switch before the dune fallback so the
  gradle daemon env (no _opam/bin on PATH, no LOGSEQ_CHAT_OPAM_SWITCH)
  can still find dune and the right workspace via 'opam switch show'
- Add tests/e2e/android-local-graph-setup.yaml (bypass hosted sign-in via
  'Use without an account', create a local graph) and a 'smoke' module
  running signed-out + setup + capture on a fresh install — no backend
  credentials needed; make it the PR/push default.
- Set git insteadOf + PAT credential helper at the ios-e2e job level so
  SwiftPM can clone the ssh:// lui dependency.
… boot, cache iOS build artifacts

- seed all smoke flows via the shared graph cache (skip per-flow
  graph create + checkpoint wait; ~1-2min/flow)
- build db-sync in a detached background process while the iOS app
  and Android APK/emulator run; a wait step polls the port
- boot the iOS simulator before the app build (bootstatus gates later)
- cache apple/.build (SwiftPM + DerivedData) keyed on Package.resolved
The 'Choose a graph' runFlow condition evaluated while the graph picker
was still loading, so graph creation was skipped and the final
button.sidebar wait timed out. Now wait for button.sidebar (already in
app) or 'Choose a graph' (picker) explicitly before deciding.
logseq/lui#34 fixes RenderFlex asserting 'children have non-zero flex
but incoming constraints are unbounded' when a grow child sits inside a
shrink-wrap column/row — the cause of the signed-out white screen in
Android e2e.
@tiensonqin
tiensonqin merged commit c4c0039 into main Sep 24, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant