gitmote

gitmote  ·  history  ·  branches

docs: plan "easy to operate" — one portable container, minimal env

150e7dfe6ae36b7d4b563059a9e54a53a3f910fc
Atanas Minev <atmin@protonmail.com> · 2026-07-05 00:50

diff --git a/tasks/README.md b/tasks/README.md
index 7424f27..e0aa476 100644
--- a/tasks/README.md
+++ b/tasks/README.md
@@ -4,16 +4,46 @@ Active and upcoming work. One file per unit of work, `Spec → Current → Chang
Verify`, **deleted once it lands** (commits are the record — this list never
becomes a changelog).
-**The frontier is currently clear.** The **gitmote can host itself** milestone is
-reached: it runs on Scaleway at `gitmote.atmin.net`, deploys on every green master
-push (GitHub Actions), survives cold starts (litestream restore), and enforces the
-single writer by a lease (s3lite `RoleAuto`), so rolling deploys are safe by
-construction.
-
-**CI has landed** — run `.github/workflows` on push, `act` self-hosted on Scaleway
-Jobs, logs, status UI, encrypted per-repo secrets. The design and rationale now
-live in [../docs/architecture/ci.md](../docs/architecture/ci.md). Known limitation:
-the Serverless Job sandbox can't build container images, so gitmote does not
-rebuild its own image — deployment stays on GitHub Actions.
-
-Speculative future directions live in [../docs/evolution/](../docs/evolution/).
+## Easy to operate — one portable container, S3 as the single source of truth
+
+The milestone (`gitmote can host itself`) is reached; the next theme is **making
+gitmote trivial to run**: `docker run ghcr.io/atmin/gitmote` with a handful of
+envs, on a laptop or on Scaleway, identically. Kill it — fine; restart it — it
+restores from S3 and continues. Localhost is a first-class deployment.
+
+Two independent chains plus a docs cap — order within a chain matters, the chains
+run in parallel:
+
+**Chain A — shrink the surface**
+- [minimal-env-replica.md](minimal-env-replica.md) — derive the DB replica from
+ the bucket (drop `GITMOTE_DB_REPLICA`); a bucket ⇒ replicate + lease (durable by
+ default, `RoleOff`-with-a-bucket retired); collapse db/cache/sock into one
+ `GITMOTE_DATA`.
+- [minimal-env-secrets.md](minimal-env-secrets.md) — auto-generate + persist
+ `GITMOTE_COOKIE_KEY` and (lazily) `WORKER_SECRET` in meta; CI master key stays
+ env-only.
+- [onboarding-bootstrap.md](onboarding-bootstrap.md) — first-run **auto-bootstrap**
+ prints the admin token once to the logs with clear instructions (no setup page,
+ no race); `bootstrap --reissue` recovers a lost token. *(needs -secrets for the
+ cookie key.)*
+
+**Chain B — the image**
+- [public-registry.md](public-registry.md) — publish `ghcr.io/atmin/gitmote` +
+ `…/gitmote-runner` publicly; deploy from GHCR; kill the by-hand runner push.
+- [dogfood-make.md](dogfood-make.md) — `make image` / `make prod` / `make publish`;
+ `make prod` runs the real image against dev MinIO. *(needs -replica for full
+ shared state, and -registry for the tags.)*
+
+**Cap**
+- [operate-docs.md](operate-docs.md) — `docker run` quickstart (incl. "grab the
+ token from the logs") + CI substrate auto-detection docs. *(last — describes the
+ final surface.)*
+
+## Later
+
+- **URL redesign** — unify `/-/tree/` vs `/-/blob/` (or rewrite relative links in
+ rendered markdown) so in-repo markdown links resolve correctly and carry the
+ ref. (A relative link on a tree page currently resolves to `/-/tree/<file>` →
+ "empty tree".)
+
+Speculative directions live in [../docs/evolution/](../docs/evolution/).
diff --git a/tasks/dogfood-make.md b/tasks/dogfood-make.md
new file mode 100644
index 0000000..a8f682d
--- /dev/null
+++ b/tasks/dogfood-make.md
@@ -0,0 +1,43 @@
+# Dogfood Make targets — build, run, publish the image
+
+Part of [easy to operate](README.md). Operator ergonomics for the container
+lifecycle. Depends on [minimal-env-replica](minimal-env-replica.md) (so
+`make prod` restores the *whole* forge from S3) and
+[public-registry](public-registry.md) (the image tags).
+
+## Spec
+
+Three targets, distinct from `make build` (which stays the Go binaries):
+
+- `make image` — `docker build` the server (and runner) image, tagged
+ `ghcr.io/atmin/gitmote:$(VERSION)` / `…/gitmote-runner:$(VERSION)`.
+- `make prod` — run **that image** against the dev MinIO, sharing the `gitmote`
+ bucket so it sees the same state as `make dev`, on `:8080`. This is the local
+ dogfood of the exact prod deployment — proof the container is portable and
+ restores from S3.
+- `make publish` — `docker push` the image(s) to GHCR (after `docker login
+ ghcr.io`). CI publishes on master; this is the manual / first-time path.
+
+## Current
+
+The Makefile builds Go binaries (`make build`) and runs gitmote **natively** via
+`make dev`. There is no way to exercise the actual container image locally, so the
+"portable container, localhost is valid" claim is untested outside prod.
+
+## Change
+
+- Add `image` / `prod` / `publish` targets. `make prod` reaches MinIO over the dev
+ compose network, not the host port: `--network gitmote-dev_default` (resolve the
+ name robustly — `podman compose` may name it differently),
+ `GITMOTE_S3_ENDPOINT=http://minio:9000`, `GITMOTE_S3_BUCKET=gitmote`, MinIO creds.
+- Reuse the dev MinIO (bring it up if needed).
+
+## Verify
+
+- `make image && make prod` boots the built image against the dev MinIO, shares
+ `make dev`'s state (meta + objects, post-replica-task), and serves the UI on
+ `:8080` — the portability / restore-from-S3 proof.
+- Run **one at a time** with `make dev`: the lease makes the second instance a
+ follower that 503s reads (correct single-writer behavior) — verify that's what
+ happens, not a crash.
+- `make publish` pushes to GHCR (manual login).
diff --git a/tasks/minimal-env-replica.md b/tasks/minimal-env-replica.md
new file mode 100644
index 0000000..b19a7a3
--- /dev/null
+++ b/tasks/minimal-env-replica.md
@@ -0,0 +1,62 @@
+# Shrink the server env — derive the replica, collapse the data paths
+
+Part of [easy to operate](README.md). Cut required and optional envs so a run is
+a handful of vars, and make "stateless, restores-on-restart" the default.
+
+## Spec
+
+- **Derive the replica from the bucket.** With `GITMOTE_S3_BUCKET` set, gitmote
+ replicates metadata to S3 and runs the single-writer lease **without a second
+ env**. `GITMOTE_DB_REPLICA` becomes an optional override, not a requirement.
+- **A bucket means durability, always.** Once the replica is derived, "bucket set
+ but not replicating" (`RoleOff` with a bucket) has no real use — ephemeral/test
+ runs use the in-memory store (no bucket). So `RoleOff` stops being a
+ bucket-mode: a bucket ⇒ `RoleAuto` (replicate + lease). This is a behavior
+ change for `make dev` / `e2e-local` (they gain replication + a lease); it is the
+ point — it's what lets `make prod` share state and restore from S3.
+- **One data dir.** Collapse `GITMOTE_DB` / `GITMOTE_CACHE` / `GITMOTE_SOCK` into
+ a single `GITMOTE_DATA` (default a temp dir): db `= $DATA/meta.sqlite3`, cache
+ `= $DATA/cache`, sock `= $DATA/gitmote.sock`. One `-v ./data:/data` for the
+ whole container. The three specific vars can stay as overrides if trivial, but
+ the documented knob is one.
+- **Fewer knobs.** Hardcode the listen address to `:8080` — drop `GITMOTE_ADDR`
+ (map ports at the container). Retire `GITMOTE_S3_PREFIX` as a documented knob
+ (keep the code default `""` so an existing prefixed bucket still works, but it
+ leaves the surface — it only mattered for sharing a bucket across apps).
+
+## Current
+
+[`metaConfigFromEnv`](../cmd/gitmote/main.go) sets `Role` only when
+`GITMOTE_DB_REPLICA` is non-empty → `RoleOff` otherwise. It's called **twice** —
+`RoleWriter` for the bootstrap CLI and `RoleAuto` for the server — so the
+derivation lands in one place and fixes both. Prod runs
+`GITMOTE_S3_PREFIX=objects/` with the replica at `s3://gitmote/meta` — meta and
+objects are **siblings** under the bucket root. Path vars default individually
+(`GITMOTE_DB` → `gitmote.sqlite3`, cache/sock → temp).
+
+## Change
+
+- In `metaConfigFromEnv`, when `GITMOTE_DB_REPLICA` is empty but the bucket is
+ set, derive `replica = s3://{bucket}/meta`. **Derive from the bucket only — not
+ bucket + prefix.** `s3://{bucket}/{prefix}meta` would be a *different* path and
+ orphan the existing replica; the sibling layout (`objects/` + `meta`) is
+ deliberate. (No prod data is at risk today, but get the layout right so it
+ stays migration-free.)
+- Drop the `RoleOff`-with-a-bucket path; a bucket always yields `RoleAuto`.
+- Add `GITMOTE_DATA` and derive the three paths from it; keep the explicit vars as
+ overrides.
+- Replace `envOr("GITMOTE_ADDR", ":8080")` with a `:8080` constant (drop the flag
+ + env). Stop reading/documenting `GITMOTE_S3_PREFIX` in the ops surface (leave
+ `store.NewS3FromEnv`'s default `""` intact for back-compat).
+- Update `scripts/dev.sh` (drop the explicit replica + the path vars; use
+ `GITMOTE_DATA=data`), and the `docs/ops.md` / `Dockerfile` env docs.
+
+## Verify
+
+- Unit test: derivation is `s3://{bucket}/meta` for any prefix (assert the prefix
+ does **not** leak into the replica path); an explicit `GITMOTE_DB_REPLICA` still
+ overrides.
+- `make e2e-restore` (cold-start restore) and `make e2e-local` stay green with the
+ now-default replication.
+- `GITMOTE_DATA=./data` alone places db/cache/sock under it.
+- `gofmt`/`golangci-lint`/`go test ./...` clean.
diff --git a/tasks/minimal-env-secrets.md b/tasks/minimal-env-secrets.md
new file mode 100644
index 0000000..867f221
--- /dev/null
+++ b/tasks/minimal-env-secrets.md
@@ -0,0 +1,55 @@
+# Auto-generate and persist COOKIE_KEY and WORKER_SECRET
+
+Part of [easy to operate](README.md). Make the UI and local CI work with **zero
+secret config**, stable across scale-to-zero.
+
+## Spec
+
+`GITMOTE_COOKIE_KEY` (signs session cookies) and `WORKER_SECRET` (authenticates
+the CI report API) should not be required. Generate each on demand, **persist in
+meta** (survives restart / scale-to-zero via S3 restore), reuse thereafter. An
+env, when set, wins. The CI-secrets master key (`GITMOTE_CI_SECRET_KEY_V<n>`)
+stays **env-only** — it must not live in the replica (safety.md §5); unchanged.
+
+Persist, not ephemeral: a per-boot key would log everyone out on each idle→wake
+and orphan in-flight CI — wrong for a scale-to-zero container.
+
+Scope tightly: this is get-or-create for **exactly these two named keys**, not a
+general key–value abstraction (CLAUDE.md §3 — no speculative generality).
+`WORKER_SECRET` is generated **lazily** — only when a CI trigger is configured
+(it's meaningless with CI off), so a plain no-CI run generates only the cookie key.
+
+## Current
+
+`GITMOTE_COOKIE_KEY` absent → [`buildUI`](../cmd/gitmote/main.go) returns nil, the
+UI is disabled. `WORKER_SECRET` absent → the CI trigger is `NoopTrigger` (or a
+fail-fast when `SCW_CI_JOB_DEFINITION_ID` is set). Both are hand-supplied today.
+The schema is `CREATE TABLE IF NOT EXISTS` + an additive `migrate()` (meta.go), so
+adding a `server_secrets` table needs no migration infra.
+
+## Change
+
+- Add a `server_secrets(name TEXT PRIMARY KEY, value BLOB)` table and a narrow
+ `GetOrCreateSecret(ctx, name, gen func() ([]byte, error))` in
+ [`internal/meta`](../internal/meta), leader-written like any row.
+- At startup resolve each: env → else meta get-or-create. The **leader** generates
+ + persists on first boot (uncontended cold start is leader); later boots / a
+ follower read the restored value.
+- Wire the cookie key into `buildUI` (UI always on) and the worker secret into the
+ dispatcher + report API — generating the worker secret only when a trigger is
+ configured.
+- Note in `safety.md`: these two now live in the replica (distinct from the
+ env-only CI master key), and why it's acceptable (the replica already holds
+ users/tokens/ACLs; sessions are short-lived; the worker secret only authorizes
+ CI report submission).
+
+## Verify
+
+- UI works with **no** `GITMOTE_COOKIE_KEY`; a session stays valid across a
+ process restart (same key restored), resetting only if meta/S3 is wiped.
+- **Cloud propagation:** an auto-generated `WORKER_SECRET` reaches the Scaleway
+ runner — it's injected at trigger time (`env["WORKER_SECRET"] = d.secret` in
+ dispatcher.go), not baked into the job definition. Add a test asserting the
+ resolved secret is the one the dispatcher injects and the report API validates.
+- An explicit env overrides the persisted value.
+- `gofmt`/`golangci-lint`/`go test ./...` clean; golden + failure paths.
diff --git a/tasks/onboarding-bootstrap.md b/tasks/onboarding-bootstrap.md
new file mode 100644
index 0000000..0d4b7e4
--- /dev/null
+++ b/tasks/onboarding-bootstrap.md
@@ -0,0 +1,61 @@
+# First-run auto-bootstrap + token recovery
+
+Part of [easy to operate](README.md). A fresh empty-bucket `docker run` should be
+usable without a second command, and a lost token must be recoverable without
+nuking the bucket. Lands after [minimal-env-secrets](minimal-env-secrets.md) (the
+cookie key must resolve for the UI).
+
+## Spec
+
+- **Auto-bootstrap on first run.** When this instance is the leader and **no admin
+ exists**, create the admin (handle from `GITMOTE_ADMIN_HANDLE`, default
+ `admin`), mint a token, and print it **once** to the logs behind an unmissable,
+ self-explanatory banner: what it is, how to sign in (paste at `/login`), how to
+ clone/push, that it won't be shown again, and how to re-mint if lost. No web
+ setup page, no unauthenticated surface, no first-run race.
+- **No initial repo.** Drop the bootstrap-creates-a-repo step; repos are made in
+ the UI in two clicks. Removes the `-repo` / `-default-branch` inputs.
+- **Recovery via `bootstrap --reissue`.** Mints a fresh token for the existing
+ admin and prints it once — the safe answer to a lost token: whoever can run the
+ container against the bucket already has total infra control, so out-of-band
+ re-mint is the correct authz boundary. It needs the writer lease, so run it
+ while the server is idle/stopped (scale-to-zero frees the lease). No token is
+ ever stored at rest — meta keeps only the hash.
+
+Why logs, not a page: the log is operator-only (same trust boundary as the env
+and bucket), so a one-time token there is strictly simpler than a new HTTP
+surface — no gating, no race, no claim-code machinery.
+
+## Current
+
+`gitmote bootstrap` is a separate CLI subcommand (needs `-handle`/`-repo`),
+refuse-on-exists ([internal/bootstrap](../internal/bootstrap), `AdminExists()`
+gates it), mints a token once. A fresh server is unusable until someone runs it;
+a lost token means lock-out (re-bootstrap refuses) with only "nuke the bucket" as
+an escape. `data/dev-token` is a local dev convenience, not a prod model.
+
+## Change
+
+- On server startup, after `meta.Open`, when leader && `!AdminExists()`: run
+ `bootstrap.Run` and log the token with the banner. Subsequent boots: admin
+ exists → skip. (A follower can't reach this — a first-ever boot is uncontended,
+ so it is the leader.)
+- Make the repo optional in `bootstrap.Run`; default the admin handle to `admin`.
+- Add a `--reissue` flag (or mode) to the `bootstrap` CLI: with an admin present,
+ mint a fresh token for it and print once (reuses the existing RoleWriter/lease
+ machinery — no second subcommand).
+- Update `scripts/dev.sh` to rely on auto-bootstrap (drop the manual bootstrap
+ step + the `data/dev-token` dance; read the token from the server log or just
+ re-issue).
+
+## Verify
+
+- Empty bucket: the first `docker run` logs a token + clear instructions; signing
+ in at `/login` with it works.
+- Second boot: no new token (admin exists); existing sessions/tokens still work.
+- `bootstrap --reissue` mints a working fresh token for the existing admin without
+ touching other data; requires the writer lease.
+- Only the hash is stored (assert no raw token in meta/S3).
+- Docs note: the token transits the logs (operator-visible) — rotate it after
+ first login by minting your own and revoking `bootstrap`.
+- `gofmt`/`golangci-lint`/`go test ./...` clean; golden + failure paths.
diff --git a/tasks/operate-docs.md b/tasks/operate-docs.md
new file mode 100644
index 0000000..8f920b5
--- /dev/null
+++ b/tasks/operate-docs.md
@@ -0,0 +1,46 @@
+# Operate docs — quickstart + CI substrate
+
+Part of [easy to operate](README.md). Lands after the env cuts + GHCR, so the docs
+describe the reduced surface.
+
+## Spec
+
+A newcomer copies one `docker run` and has a working forge. Make true in the docs:
+
+1. **Run quickstart** — the minimal env set (`GITMOTE_S3_BUCKET`, the three
+ `AWS_*`, `GITMOTE_S3_ENDPOINT` if non-AWS; optionally `GITMOTE_DATA` for a
+ mounted volume) and a copy-paste `docker run ghcr.io/atmin/gitmote`. State that
+ S3 is the single source of truth and the container is disposable
+ (kill/restart restores), and that **the first-run admin token is printed once
+ to the logs** — where to look, how to sign in, and how to re-mint
+ (`bootstrap --reissue`) if lost.
+2. **CI substrate auto-detection** — how gitmote picks its runner: present
+ `SCW_CI_JOB_DEFINITION_ID` → Scaleway Serverless Jobs; else
+ `GITMOTE_URL`+`WORKER_SECRET` → local `act` (needs docker/podman on the host);
+ else CI records runs but does not execute. State the docker/podman prerequisite
+ explicitly.
+
+## Current
+
+`docs/ops.md` documents the full pre-cut env list and the Scaleway-first deploy;
+no minimal quickstart, and the CI substrate fallback is undocumented (it's in
+[ci.md](../docs/architecture/ci.md)'s runner table but not as operator guidance).
+`README.md` has no "run it" path.
+
+## Change
+
+- Add a **Quickstart** to `README.md` (or a short `docs/run.md`): the minimal env
+ set, `docker run ghcr.io/atmin/gitmote …`, and the "grab the token from the
+ logs" first-login step.
+- Refresh `docs/ops.md`'s env table: mark removed (`GITMOTE_DB_REPLICA`, the
+ individual `GITMOTE_DB`/`CACHE`/`SOCK` → `GITMOTE_DATA`), auto-managed
+ (`GITMOTE_COOKIE_KEY`, `WORKER_SECRET`), and feature-gated (`SCW_*`,
+ `GITMOTE_URL`, `GITMOTE_CI_SECRET_KEY_V<n>`); keep the required core small.
+- Add a short **CI substrate** section (auto-detection rules + docker/podman note)
+ and cross-link `ci.md`.
+
+## Verify
+
+- The env tables match the code after the earlier tasks (no stale vars).
+- Following the quickstart from a clean machine yields a running, pushable forge.
+- Docs and code agree (CONTRIBUTING's "fix one in the same change").
diff --git a/tasks/public-registry.md b/tasks/public-registry.md
new file mode 100644
index 0000000..fd041f3
--- /dev/null
+++ b/tasks/public-registry.md
@@ -0,0 +1,47 @@
+# Public GHCR images; deploy from public
+
+Part of [easy to operate](README.md). Make the image runnable "from the
+internet" and stop hand-pushing the runner. Independent of the env-shrink tasks
+(can proceed in parallel).
+
+## Spec
+
+`gitmote` and `gitmote-runner` are published **publicly** to GHCR
+(`ghcr.io/atmin/gitmote`, `…/gitmote-runner`). `docker run ghcr.io/atmin/gitmote`
+works anywhere. Scaleway deploys by pulling the public image (its Serverless
+Container + the CI Job definition both reference GHCR). This is also the pragmatic
+self-deploy path: build where a real daemon exists (GitHub Actions, or locally),
+push public, and "deploy" is a retag + `scw container update` — no in-Job build
+(the sandbox can't).
+
+## Current
+
+`.github/workflows/ci.yml` builds `rg.fr-par.scw.cloud/atmin/gitmote` (private
+Scaleway registry) and `scw container update`s it. The **runner** image is built
+and pushed **by hand** ([ci.md](../docs/architecture/ci.md)); the Job definition
+points at `rg.fr-par.scw.cloud/atmin/gitmote-runner:master`.
+
+## Change
+
+- Deploy job: build + push `ghcr.io/atmin/gitmote:{sha,master}` (public), point
+ `scw container update image=ghcr.io/atmin/gitmote:<sha>`.
+- Add a **runner image** build+push to CI (on `Dockerfile.runner` change / on
+ release), publishing `ghcr.io/atmin/gitmote-runner` — removes the manual push.
+- Update the `scw jobs definition` image to the GHCR runner path (one-time, doc
+ in ops.md).
+
+## Verify
+
+- A bare `docker run --rm -e GITMOTE_S3_BUCKET=… -e AWS_… ghcr.io/atmin/gitmote`
+ boots and serves `/healthz`.
+- A push to master builds + pushes GHCR and Scaleway redeploys from it (container
+ image SHA flips to the GHCR ref).
+- A cloud CI run pulls the GHCR runner image and reports green.
+
+## Gotcha
+
+New GHCR packages default to **private** — the package must be flipped to
+**public** once by hand (GitHub → package settings) before Scaleway can pull it
+anonymously. Otherwise the first deploy fails on an image pull the code can't
+diagnose. Publishing from Actions needs `packages: write` (built-in
+`GITHUB_TOKEN`).