gitmote

gitmote  ·  history  ·  branches

docs: add task frontier toward self-hosting

ca01906267d390973085c2fdfc76b1304d9cddea
Atanas Minev <atmin@protonmail.com> · 2026-07-02 15:39

diff --git a/tasks/01-project-scaffold.md b/tasks/01-project-scaffold.md
new file mode 100644
index 0000000..26c1a95
--- /dev/null
+++ b/tasks/01-project-scaffold.md
@@ -0,0 +1,32 @@
+# 01 — Project scaffold & gates
+
+Depends on: nothing.
+
+## Spec
+
+Establish the build/test/lint gates promised in
+[CONTRIBUTING.md](../CONTRIBUTING.md) so every later task can land green, plus a
+minimal server binary that starts, reports version/health, and shuts down
+cleanly.
+
+## Current
+
+Docs only — no `go.mod`, no code, no Makefile.
+
+## Change
+
+- `go.mod` (module `github.com/atmin/gitmote`, Go pinned to a recent stable).
+- `cmd/gitmote/main.go` — env/flag config, structured logging, an HTTP server
+ with `GET /healthz` and `GET /version`, graceful shutdown on SIGINT/SIGTERM.
+- `Makefile` — the single source of truth: `fmt` (gofmt/goimports), `lint`
+ (golangci-lint, includes vet), `test` (`go test ./...`), `build`, and
+ `all = lint test build`.
+- `.golangci.yml`; `.github/workflows/ci.yml` calling the same Make targets;
+ `scripts/pre-commit` running `make fmt lint test`.
+
+## Verify
+
+- `make all` is green from a clean checkout; CI runs the Make targets and passes.
+- `go run ./cmd/gitmote` serves `200` on `/healthz` — asserted with an
+ `httptest` unit test.
+- Non-breaking: this is the baseline; nothing to regress.
diff --git a/tasks/02-s3-object-store.md b/tasks/02-s3-object-store.md
new file mode 100644
index 0000000..e87dda3
--- /dev/null
+++ b/tasks/02-s3-object-store.md
@@ -0,0 +1,33 @@
+# 02 — S3 object store
+
+Depends on: 01.
+
+## Spec
+
+Durable, content-addressed storage for git objects and packs in S3, per the S3
+layout in [storage.md](../docs/architecture/storage.md). PUT is idempotent
+(re-PUT of the same hash is a no-op) — the substrate for the content-before-pointer
+invariant in [safety.md](../docs/architecture/safety.md).
+
+## Current
+
+Nothing yet.
+
+## Change
+
+- `internal/store/store.go` — the `Store` interface (the storage contract):
+ Put / Get / Exists / List over keys under `{repo}/objects/…` and
+ `{repo}/objects/pack/…`.
+- `internal/store/s3.go` — S3 implementation (`aws-sdk-go-v2`; the arch notes
+ the SDK directly, with `rclone` as a zero-code fallback). Config via env.
+- `internal/store/mem.go` — in-memory implementation for tests, kept in lockstep
+ with the S3 impl (CONTRIBUTING: both satisfy `Store`).
+- A shared conformance test suite exercised against both impls.
+
+## Verify
+
+- Conformance suite passes for `mem` always, and for `s3` against MinIO when
+ available (gated by env).
+- Re-PUT of the same key is idempotent; `Exists`/`List` behave at prefix
+ boundaries.
+- Non-breaking: a library with no route wired in yet.
diff --git a/tasks/03-metadata-s3lite.md b/tasks/03-metadata-s3lite.md
new file mode 100644
index 0000000..a9af41d
--- /dev/null
+++ b/tasks/03-metadata-s3lite.md
@@ -0,0 +1,33 @@
+# 03 — Metadata layer (s3lite)
+
+Depends on: 01.
+
+## Spec
+
+All mutable forge state — refs, repos, users, tokens, ACLs — in s3lite, per the
+schema in [storage.md](../docs/architecture/storage.md). Refs are the source of
+truth; a ref update is a single-transaction compare-and-swap
+([safety.md](../docs/architecture/safety.md) §3). Durability (WAL → S3) is
+s3lite's own concern.
+
+## Current
+
+Nothing yet. s3lite is a working dependency to import.
+
+## Change
+
+- Add `github.com/atmin/s3lite`; open it as `*sql.DB` inside a `Metadata` type.
+- `internal/meta/schema.sql` + migrations run automatically on Open (idempotent)
+ — the tables from storage.md: `repos`, `refs`, `users`, `tokens`, `ssh_keys`,
+ `acls`.
+- `internal/meta/` query layer: `ListRefs`, `CASRef(repo, name, old, new)` (one
+ transaction; a multi-ref variant for atomic push), repo/user/token CRUD, ACL
+ lookup. Tokens stored **hashed**, never raw.
+- Test seed helpers (insert repo/user/token/refs directly) for later tasks.
+
+## Verify
+
+- Migrations create the schema on a fresh (empty) s3lite; re-open is a no-op.
+- CAS unit tests: success on match; reject + rollback on mismatch; multi-ref
+ is all-or-nothing.
+- Non-breaking: DB layer + tests, no route yet.
diff --git a/tasks/04-repo-materialization.md b/tasks/04-repo-materialization.md
new file mode 100644
index 0000000..2d1e2d7
--- /dev/null
+++ b/tasks/04-repo-materialization.md
@@ -0,0 +1,34 @@
+# 04 — Repo materialization / cache
+
+Depends on: 02, 03.
+
+## Spec
+
+Produce a valid on-disk **bare** repo for an operation — a disposable cache,
+never the source of truth ([storage.md](../docs/architecture/storage.md) →
+ephemeral disk). Refs come from s3lite; objects are hydrated from S3 into the
+closure the operation needs.
+
+## Current
+
+Nothing yet.
+
+## Change
+
+- `internal/repo/materialize.go` — given a repo name: create a temp bare repo
+ (`git init --bare`), write refs from s3lite, hydrate objects from the `Store`.
+- **Hydration policy: full-hydrate for the MVP** — a write hydrates the target
+ branch's full history (needed for the fast-forward / connectivity check), a
+ read hydrates the closure it serves. Partial-clone / promisor is a later
+ optimization (see
+ [notes/object-hydration.md](../docs/notes/object-hydration.md)); gitmote's own
+ repo is tiny, so full-hydrate is the safe, simple first cut.
+- Per-repo cache dir with rebuild-on-miss; eviction is best-effort (disposable).
+
+## Verify
+
+- Given a seeded repo (S3 objects + s3lite refs), materialize then assert with
+ real git: `git -C <dir> rev-parse <ref>` and `git cat-file --batch-check`
+ resolve; `git fsck` is clean.
+- A cold rebuild after deleting the cache dir reproduces an equivalent repo.
+- Non-breaking: internal helper, no route yet.
diff --git a/tasks/05-read-path-clone.md b/tasks/05-read-path-clone.md
new file mode 100644
index 0000000..000dcb2
--- /dev/null
+++ b/tasks/05-read-path-clone.md
@@ -0,0 +1,29 @@
+# 05 — Read path: clone / fetch
+
+Depends on: 04.
+
+## Spec
+
+Serve smart-HTTP `git-upload-pack` so `git clone` / `fetch` work, per the read
+path in [request-flows.md](../docs/architecture/request-flows.md). No lock; refs
+advertised from s3lite, object closure hydrated from S3.
+
+## Current
+
+Only `/healthz` and `/version` exist (task 01).
+
+## Change
+
+- Routes: `GET /{repo}/info/refs?service=git-upload-pack` (advertise from
+ s3lite) and `POST /{repo}/git-upload-pack`.
+- Materialize (task 04), then delegate protocol work to stock git via
+ `net/http/cgi` invoking `git http-backend` (arch: wrap `http-backend`). Set
+ `GIT_PROJECT_ROOT` / `PATH_INFO` to the materialized repo.
+- Unauthenticated for now — auth lands in task 06.
+
+## Verify
+
+- Against a running server with a seeded repo, a real
+ `git clone http://localhost/{repo}` succeeds and `git fsck` on the clone is
+ clean; `git fetch` after a seeded update pulls the new commits.
+- Non-breaking: adds read-only endpoints; existing routes untouched.
diff --git a/tasks/06-auth-pat-acl.md b/tasks/06-auth-pat-acl.md
new file mode 100644
index 0000000..ef75065
--- /dev/null
+++ b/tasks/06-auth-pat-acl.md
@@ -0,0 +1,32 @@
+# 06 — Auth: PATs + per-repo ACLs
+
+Depends on: 03, 05.
+
+## Spec
+
+Authenticate smart-HTTP with bearer **personal access tokens** (hashed in
+`tokens`) and authorize **per-repo** from `acls` on every request, per
+[auth.md](../docs/architecture/auth.md).
+
+## Current
+
+The read path (task 05) is open to anyone.
+
+## Change
+
+- `internal/auth` — PAT mint (return the raw token once, store only its hash) +
+ verify with constant-time compare; update `last_used`.
+- A request guard resolving `Authorization: Bearer …` → user, and an
+ `authorize(repo, user, perm)` helper reading `acls`.
+- Apply `read` authorization to the task-05 routes. `401` for missing/invalid
+ token, `403` for insufficient permission, using git's expected auth challenge
+ so `git` and credential helpers behave.
+
+## Verify
+
+- Golden: clone with a valid `read` token succeeds.
+- Rejections (not optional): no token → 401; unknown/bad token → 401; valid
+ token lacking `read` on the repo → 403.
+- Non-breaking: the read path now requires auth by design; read tests are
+ updated to seed a token + ACL. No prior external consumers exist to break
+ (design stage).
diff --git a/tasks/07-write-path-cas.md b/tasks/07-write-path-cas.md
new file mode 100644
index 0000000..ea36b5e
--- /dev/null
+++ b/tasks/07-write-path-cas.md
@@ -0,0 +1,43 @@
+# 07 — Write path: push (the CAS)
+
+Depends on: 04, 03, 06.
+
+## Spec
+
+Serve `git-receive-pack` so `git push` works, with the safety-critical ordering
+**objects durable in S3 first, ref CAS in s3lite second** — the write path and
+all four points in [request-flows.md](../docs/architecture/request-flows.md) and
+[safety.md](../docs/architecture/safety.md). Resolves the push-hook channel
+([notes/push-hook-channel.md](../docs/notes/push-hook-channel.md)).
+
+## Current
+
+Read path only. No way to write.
+
+## Change
+
+- Routes: `GET /{repo}/info/refs?service=git-receive-pack` and
+ `POST /{repo}/git-receive-pack`, gated by `write` ACL (task 06).
+- A per-repo in-process mutex serializes writes; materialize target history
+ (full-hydrate) before spawning `receive-pack` and piping the request body.
+- Install a `pre-receive` hook into the materialized repo that RPCs the parent
+ over a unix socket — **decide:** socket path passed via the hook's env, hook
+ authenticated by a per-request nonce. The parent (the sole writer) PUTs
+ quarantined objects to S3 (`$GIT_QUARANTINE_PATH`) — content before pointer —
+ then runs the per-ref CAS in one s3lite transaction and returns a verdict the
+ hook turns into its exit code.
+- On `ok`, receive-pack migrates the quarantine; report-status flows back.
+
+## Verify
+
+- Golden: a real `git push` of new commits succeeds; a subsequent clone
+ (task 05) returns them.
+- Safety tests — the point of this task:
+ - **non-fast-forward is rejected** and leaves refs unchanged.
+ - **concurrent-push CAS**: two pushes racing the same ref → exactly one wins,
+ the other is rejected; no lost update.
+ - **atomic multi-ref**: a push updating two refs where one fails rolls back
+ both.
+ - **ordering invariant**: inject a ref-CAS failure after the object PUT → S3
+ holds orphan objects (harmless), the ref never advanced (no corruption).
+- Non-breaking: adds write endpoints; the read path is unaffected.
diff --git a/tasks/08-bootstrap.md b/tasks/08-bootstrap.md
new file mode 100644
index 0000000..b4da31e
--- /dev/null
+++ b/tasks/08-bootstrap.md
@@ -0,0 +1,28 @@
+# 08 — Bootstrap
+
+Depends on: 07.
+
+## Spec
+
+Get from an empty bucket + empty s3lite to a usable instance: schema, a first
+**admin** user + token, and a first repo — per
+[notes/bootstrap.md](../docs/notes/bootstrap.md).
+
+## Current
+
+Schema is created on Open (task 03), but there is no entry path to create the
+first admin/token/repo — a chicken-and-egg with token auth.
+
+## Change
+
+- `gitmote bootstrap` subcommand (arg-dispatched in `main`, no server): ensure
+ schema, create an admin user, mint + print a token **once**, create an initial
+ repo. Idempotent where safe; refuse to clobber an existing admin.
+- Document the one-time flow in README / ops.
+
+## Verify
+
+- From a fresh MinIO bucket: run `bootstrap`, then use the printed token to
+ `git push` and `git clone` the initial repo end-to-end.
+- Re-running bootstrap neither duplicates nor overwrites the admin.
+- Non-breaking: new subcommand; the server path is unchanged.
diff --git a/tasks/09-management-web-ui.md b/tasks/09-management-web-ui.md
new file mode 100644
index 0000000..3728ab5
--- /dev/null
+++ b/tasks/09-management-web-ui.md
@@ -0,0 +1,31 @@
+# 09 — Management web UI
+
+Depends on: 06, 08.
+
+## Spec
+
+A minimal authenticated web UI for the operations you can't do over git:
+create/list repos, mint/revoke tokens, and manage per-repo ACLs. Backed by the
+same s3lite tables (the "web UI" component in
+[storage.md](../docs/architecture/storage.md)). Repo browsing and web-authored
+commits stay out of scope.
+
+## Current
+
+Management is CLI-only (bootstrap) plus direct git. No UI.
+
+## Change
+
+- Server-rendered pages (Go `html/template`) under an authed prefix — **decide:**
+ browser login as a session cookie vs. a token pasted into the UI.
+- Screens: repos (list / create / set default branch), tokens (mint shows the
+ raw token once, revoke), ACLs (grant/revoke read/write/admin per user).
+- Reuse the task-03 query layer and task-06 auth; introduce no new source of
+ truth.
+
+## Verify
+
+- Handler tests: golden path for create-repo / mint-token / grant-ACL, **plus**
+ auth rejections (unauthenticated and non-admin → denied).
+- Manual: create a repo in the UI, mint a token, push to it.
+- Non-breaking: additive routes behind auth; git paths untouched.
diff --git a/tasks/10-self-host-local.md b/tasks/10-self-host-local.md
new file mode 100644
index 0000000..8eed29d
--- /dev/null
+++ b/tasks/10-self-host-local.md
@@ -0,0 +1,29 @@
+# 10 — Self-host, locally (MinIO + Docker)
+
+Depends on: 01–09.
+
+## Spec
+
+Prove gitmote hosts itself in a local, reproducible environment: the real
+gitmote source repo pushed to, and cloned from, a running gitmote container
+backed by MinIO. Exercises both request flows and the safety model end-to-end.
+
+## Current
+
+Components pass their own tests, but nothing runs them together against real
+storage as one system.
+
+## Change
+
+- `docker-compose.yml` — MinIO + gitmote (single writer), seeded bucket.
+- `Dockerfile` — build the static binary; stock `git` present in the image.
+- `make e2e-local` — bring up the stack, run `bootstrap`, create the `gitmote`
+ repo, `git push` this working tree to it, `git clone` it into a temp dir, and
+ assert the clone matches (same `HEAD`, `git fsck` clean, identical file tree).
+
+## Verify
+
+- `make e2e-local` is green locally and in CI (a tagged / gated job).
+- Kill + restart the container mid-life: after restart the repo is still fully
+ clonable (durability via s3lite / litestream).
+- Non-breaking: adds an e2e target; unit gates unchanged.
diff --git a/tasks/11-deploy.md b/tasks/11-deploy.md
new file mode 100644
index 0000000..7cec4d4
--- /dev/null
+++ b/tasks/11-deploy.md
@@ -0,0 +1,32 @@
+# 11 — Deploy: host itself for real
+
+Depends on: 10.
+
+## Spec
+
+Run a single-writer gitmote against real object storage so the gitmote repo
+genuinely lives on it. Honor the one operational rule — **never two writer
+instances** ([safety.md](../docs/architecture/safety.md) §1).
+
+## Current
+
+Only local (MinIO) hosting exists (task 10).
+
+## Change
+
+- Provision real S3-compatible storage + a single-instance runtime; no
+ overlapping deploys of the writer (document the rule and enforce it in the
+ deploy config).
+- Point s3lite's litestream WAL target at the real bucket; verify restore on
+ cold start.
+- TLS terminated at the edge; config/secrets via env.
+- `docs/ops.md` — the canonical infra doc: resources, env vars, deploy commands,
+ the single-writer rule. Keep it updated whenever infra/env changes.
+
+## Verify
+
+- Bootstrap on the deployed instance; `git push` the gitmote repo over HTTPS and
+ `git clone` it back — the repo lives there.
+- Cold start (scale-to-zero → wake) restores metadata and serves the repo.
+- The deploy pipeline never runs two writers concurrently.
+- Milestone: **gitmote hosts itself.**
diff --git a/tasks/README.md b/tasks/README.md
new file mode 100644
index 0000000..f9dd090
--- /dev/null
+++ b/tasks/README.md
@@ -0,0 +1,21 @@
+# Tasks — the frontier
+
+Active and upcoming work toward the milestone: **gitmote can host itself.**
+One file per unit of work, `Spec → Current → Change → Verify`, **deleted once it
+lands** (commits are the record — this list never becomes a changelog).
+
+Ordered by dependency; each task lands in a non-breaking, tested state.
+
+- [01-project-scaffold.md](01-project-scaffold.md) — Go module, Makefile gates, CI, health server.
+- [02-s3-object-store.md](02-s3-object-store.md) — `Store` interface + S3 and in-memory impls for objects/packs.
+- [03-metadata-s3lite.md](03-metadata-s3lite.md) — Embed s3lite; schema, migrations, query layer, ref CAS.
+- [04-repo-materialization.md](04-repo-materialization.md) — Build ephemeral bare repos from refs + hydrated objects.
+- [05-read-path-clone.md](05-read-path-clone.md) — Smart-HTTP `upload-pack`; `git clone` works.
+- [06-auth-pat-acl.md](06-auth-pat-acl.md) — Bearer PATs + per-repo ACL enforcement.
+- [07-write-path-cas.md](07-write-path-cas.md) — `receive-pack` + pre-receive RPC + object PUT + ref CAS; `git push` works.
+- [08-bootstrap.md](08-bootstrap.md) — First admin, token, repo from an empty bucket.
+- [09-management-web-ui.md](09-management-web-ui.md) — Authenticated UI: repos, tokens, ACLs.
+- [10-self-host-local.md](10-self-host-local.md) — MinIO + Docker; e2e clone+push of the gitmote repo.
+- [11-deploy.md](11-deploy.md) — Real object storage + single-writer deploy; hosts itself for real.
+
+Milestone is done when **10** and **11** are green: the gitmote repo lives on gitmote.