Setup internals (maintainer notes)

Project (non-normative maintainer notes) · For: maintainers changing how setup works

How cotal setup works, and the cross-repo couplings it depends on. If you change one of the things in the Invariants table, update the listed siblings in the same change, or setup silently breaks for npx users.

The flow

cotal setup (implementations/cli/src/commands/setup.ts) is configure-only: it checks prerequisites, installs the Claude Code plugin, and seeds persona files, and it launches nothing: no mesh, no web dashboard, no manager, no delivery daemon, no cmux/tmux session, no demo. Starting the stack is cotal up; the dashboard is cotal web. Every file it writes is announced (→ wrote … via provenance.wrote) on stderr, or on stdout with the error when a stderr write fails. Each announcement is one line: a control character or Unicode line separator in a path, such as a newline in HOME, is printed as a \uXXXX escape. It is two-tier, gated on a machine marker. Persona seeding resolves the selected mesh root first, then uses the same .cotal/agents catalog as spawn. With no mesh it names a cwd fallback; an ambiguous or broken target refuses rather than choosing a root.

First run (no ~/.cotal/onboarded.json, or --full, or --yes) runs runFirstRun(yes):

• splash → intro → core checks (Node >= 22; locate nats-server: located, never started) → connector picker (each selected connector’s install, then its mcpServers action, which for Claude copies your user-scope MCP servers into the cotal config unless it already declares a list) → resolve and announce the persona destination → seed the generic default and optional demo personas (david/sven/me) there → offer a global install (offerGlobalInstall) → onboarded marker → a finale that lists the commands to start things (cotal up --detach, cotal web, cotal spawn …, cotal console, cotal down). Nothing is running when it returns.
• The old --auth / --open flags are gone: they set the mesh MODE at launch time, and setup no longer launches; mode is now cotal up [--open]’s concern (an unknown-option error names them, no silent no-op).

Later runs run runEnsure: resolve and announce the same destination, re-seed the default persona if it’s missing, re-offer the global install (offerGlobalInstall, same isNpx() + PATH-scan gate as first run, so a repeat npx cotal-ai setup on a machine that still lacks a durable cotal finally installs it), then print the status card (readyCard). The card is read-only probes (machineStatus/connectorStatusRows/meshStatus/recordedWebUrl/managerUp for NATS, the rows connector setup providers report, the mesh, the web dashboard, and the manager) and for anything down it prints the exact command to start it (cotal up --detach, cotal web, cotal supervise). Displaying state never depends on it; setup still launches nothing.

--skills is the status-card write: it asks installed connectors with a declared skills setup hook to reconcile their own harness, then reconciles ~/.agents/skills. The base CLI passes only the vendor-neutral skills directory, version, and state directory; connector packages own native assets and commands. It does not seed personas, install the mesh connector, offer a global install, or write the onboarded stamp. Combined with --full or --demo it is refused.

The seeded default persona has an empty active subscribe set and wildcard allowSubscribe/allowPublish ACLs. A fresh agent receives no channel traffic until it joins a channel, but can join, create, read, and post to channels on demand. The guided demo personas keep their existing welcome read and post scope. Repeat setup replaces the prior default template only when its bytes still match the shipped legacy body with allowPublish: []; any user edit makes the file ineligible and leaves it byte-identical.

Steps run in-process via runSteps (lib/steps.ts). A step can be optional (asked Y/n), carry a confirm consent prompt, or be live (it draws its own pane via lib/live-window.ts). On failure, an interactive run offers a debug handoff for each connector whose setup provider declares an assist and whose executables are on PATH (lib/assist.ts). The provider owns the harness binary and its flags; the CLI only builds the prompt. When no connector can host one, the menu says so in one line. A provider may also declare status, which returns read-only rows about what it installed. cotal status and the setup card print them, and the CLI passes only its own version and the cotal setup --skills remedy. The extensions manifest caches each connector’s setup ref, so status imports only connectors that declare a provider. The seed reconcile refreshes a seeded entry whose cache predates that ref.

The connector picker (pickConnectors) multiselects the setup connector surface (setupConnectorSurface): every connector name the live registry or the installed extension manifest advertises, materialized through the same loader the rest of the CLI uses. No connector name is written into setup.ts. setupConnectorCandidates turns that surface into choices and reads each hint off the connector’s own declarations: requires names the executables a candidate still needs on PATH, setup says whether it owns setup actions at all, and pluginRoot says whether those actions install plugin assets. A selected candidate runs its connector-owned connector action as a narrated step built by actionStep, which takes the action with the input its type declares, so the compiler checks each pairing. The provider side is checked too. Each action’s run, assist.run and status are function-typed properties, which TypeScript checks strictly, so a provider whose callback narrows the input the CLI hands it does not compile. A candidate that declares no provider is simply marked ready (OpenCode auto-wires at spawn, injecting its plugin via buildLaunch and never writing the user’s config). A selected candidate’s mcpServers action runs next the same way: the Claude provider reads the user-scope servers from Claude Code’s config and records them through the seed input the CLI hands it. That is workspace’s seedConnectorServers, which writes the operator-level cotal config under a lock and only when it declares no list for that connector, so two setups run at once record one list. The skills action runs for every present connector that declares one, selected or not, because Cotal’s authored skills are independent of mesh membership. Two experts (david, the engineer; sven, the guide) plus the operator’s own driving session (me) are written by default, and me is the persona cotal spawn me drives.

--yes forces non-interactive accept-all even on a TTY: optional plus confirm steps run (so the demo personas are written), the global install takes its default, and a failure aborts with the log path and a non-zero exit. It still launches nothing. The control plane comes up with cotal up --detach. This is the agent/CI contract; keep it working.

Invariants

Thing	Must stay in sync across	Why
Marketplace name cotal-mesh	setup.ts (materialized marketplace.json), CHANNEL_REF in extensions/connector-claude-code/src/extension.ts, repo .claude-plugin/marketplace.json	The wake channel ref plugin:cotal@cotal-mesh binds by this name
Plugin assets	the claude connector’s own src/setup.ts copy list (dist/mcp.cjs, dist/hook.cjs, .claude-plugin/plugin.json, .mcp.json, hooks/hooks.json), its package.json files field, and the release tree’s list in scripts/materialize-claude-plugin.mjs	The connector materializes its own plugin; missing or renamed assets break the install, and the base CLI has no copy of the list. The release script refuses a plugin whose .mcp.json or hooks reference a file it did not copy
Connector.pluginRoot	packages/core/src/connector.ts (contract) plus set in the claude connector’s extension.ts	A connector’s declaration that it ships installable plugin assets; the picker phrases its hint from it
BUNDLED_PKG_PREFIX	lib/nats-bin.ts ↔ the @eplightning/nats-server-* optionalDependencies in implementations/cli/package.json	The bundled NATS binary is resolved by ${prefix}-${platform}-${arch}. (Future: swap the prefix to our own @cotal-ai/nats-server-*.)
Onboard marker plus ONBOARD_VERSION	~/.cotal/onboarded.json in lib/onboard.ts; version const in setup.ts	Flips first-run vs ensure
Demo-agent format	DEMO_AGENTS in setup.ts matches the frontmatter shape read by packages/core/src/agent-file.ts (same as examples/01-lateral-coordination/agents/)	cotal spawn <name> loads these
Managed personas	each DEMO_AGENTS body carries a # managed by cotal-setup frontmatter marker; writeDemoAgent refreshes the file when the body changes, backing a marker-less (user-edited) file up to <name>.md.bak first	Edit DEMO_AGENTS plus re-run setup to update david/sven/me; delete the marker line to take ownership
DEFAULT_SERVER	packages/core/src/endpoint.ts	The address cotal up starts and the status card probes

Background processes (cotal up)

cotal up brings up the whole local stack in one place; since setup became configure-only (stage 2b), this is where the mesh and control plane start, so cotal spawn --detach / cotal_spawn find a manager right after up. The control plane comes up in cutover order: old-manager preflight → delivery daemon (auth mode only) → manager, via ensureControlPlane (lib/delivery-proc.ts). The detached processes, all stopped by cotal down:

With no explicit --server, cotal up auto-selects a free local port when the default broker address is already held by another root or an unrecorded broker; an explicit --server remains fail-loud on collision.

• Mesh: startMeshDetached (commands/up.ts) boots the background nats-server for up --detach and up -f, and writes .cotal/nats.pid and .cotal/nats.log. Foreground up runs the broker as its own child. Both modes then run one listener-ready sequence: space setup, user-auth service, mesh record, transport policy, control plane. When the space setup of a fresh boot fails, up stops the listener and removes .cotal/nats.pid before it exits.
• Delivery daemon: startDeliveryDetached / ensureDelivery (lib/delivery-proc.ts) re-execs cotal deliver detached with a pre-minted scoped delivery.creds (auth mode only, the durable backstop; open mode has none). Writes .cotal/delivery.<key>.pid and .cotal/delivery.<key>.log, where <key> is the space key (Config), so a root can serve a space per daemon.
• Manager: startManagerDetached / ensureManager (lib/manager-proc.ts) re-execs cotal supervise detached (pty runtime); it answers the control plane (cotal_spawn / cotal_despawn / cotal_persona). Writes .cotal/manager.<key>.log; managerUp(space) checks that space’s pid record for setup’s status card. The manager itself writes .cotal/manager.<key>.pid, so a supervisor started by a container entrypoint, by cron, or by hand is recorded the same way a detached cotal up is. Readers verify the recorded pid is alive and is a supervisor before trusting it (Config).

The web dashboard is not part of cotal up. It ships inside cotal-ai as the @cotal-ai/web extension and is seeded automatically by the boot reconcile, the same durable, version-locked path as the built-in connectors (SEEDED_EXTENSIONS), so it always matches the CLI version and needs no separate install. Start it with cotal web; it records .cotal/web.pid, self-registers that process with down, and is addressed as http://cotal.localhost:7799 (binds loopback; *.localhost resolves in Chrome/Firefox/Edge, Safari may need plain 127.0.0.1). Setup’s status card prints the address the dashboard recorded in .cotal/web.session once it was listening, while the PID in .cotal/web.pid is alive. A .cotal/web.pid that exists but cannot be read is named on the row as pidfile unreadable, an unreadable .cotal/web.session as web.session unreadable, and the rest of the card still prints.

All recorded local processes self-register local-process descriptors. Bare cotal down resolves the full set and stops it in dependency order; cotal down manager (or another component name) selects only that descriptor. Installed extensions cache their contributed registry keys, so the base CLI does not hardcode optional package pidfiles.

Each recorded pidfile also carries a sibling <pidfile>.identity pin (pid plus the process’s start, where the OS reports one). The pin proves that the live process is still the recorded one. A reused pid and a torn or unreadable pin are refused and preserved. A pre-pin record warns and is signalled so an upgraded CLI can stop a stack launched by the previous version; the next launch writes a pin. A record clears only once death is confirmed.

All re-execs resolve this CLI via selfArgv() / selfCotal() (lib/self-exec.ts) = [node, ...loaderFlags, entry] (tsx loader in dev, compiled JS in prod), so they never need cotal on PATH; the stack comes up identically via npx, npm i -g, and a dev clone.

selfArgv() throws unless process.argv[1] resolves, through any symlink, to the bin the cotal-ai package declares (dist/cotal.js) or to the cotal.ts beside that package’s manifest (a checkout’s bin/cotal.ts). Started from any other file, such as a smoke suite under tsx, a re-exec would run that file again with a subcommand it ignores, and a file that reaches a starter on load would spawn its own successor (#1629). The auth, manager and delivery starters ask before they touch a pidfile or a log, and seedOne asks before it writes its cursor, stages a payload or writes its child marker, so a refusal on those paths leaves none of them behind.

For ergonomics only, an npx run with no global cotal offers to npm i -g cotal-ai (offerGlobalInstall, pinned to the running version): gated on isNpx() plus a PATH scan (cotalOnPath(), not an exec probe, since cotal --version is not a real command). The interactive prompt defaults to yes, the non-interactive path (--yes or no TTY) takes the default, and a failed install is non-fatal (warn plus manual command). The same self-exec.ts exposes displayCmd(), the prefix (cotal / npx cotal-ai / pnpm cotal) used in the status-card hints so they match how you ran it.

Built-in connectors are seeded extensions

The first-party connectors (claude, opencode, codex, hermes, pi) are not static-imported by the binary. The composition root (bin/cotal.ts) registers no connector; they self-register only when imported, and they are imported only once installed. On the first real command of each boot the CLI seeds them through the same cotal ext add path a third party uses, so they are ordinary extensions you can cotal ext remove. Code lives in implementations/cli/src/seed/; the entry is reconcileSeededConnectors(), gated in runCli before the manifest overlay so ext seed --repair survives a corrupt manifest.

What ships where. The connectors are devDependencies of cotal-ai (not runtime deps), and a prepack step (bin/scripts/copy-seeded-connectors.mjs) npm packs each into bin/seeded-connectors/<name>/ (honoring each connector’s own files), added to the package files. SEEDED_EXTENSIONS (@cotal-ai/workspace) is the shared list: the connectors plus web. The prepack asserts that every bundled payload’s name and version match the umbrella (the fixed changeset group keeps them lockstep), so a version-skewed payload can never be published; web also emits dist/web/vendor/vendor-manifest.json (name/version/license/sha512) as the auditable inventory of its vendored browser libs (marked/DOMPurify ship as opaque dist bytes, not runtime deps). seed/paths.ts:shippedSourceDir resolves the live extensions/<pkg> dir in a source checkout and <cotal-ai>/seeded-connectors/<name> in a published install. The reconcile copies that payload into the durable store seed/store/<version>/<name>. The version is validated as one safe path segment, and the destination is checked to stay inside the store before anything is written. ext add --install-links reifies the file: dep from THAT stable path (a volatile source would fail to re-reify); ext add then junction-links each @cotal-ai/* peer to the binary’s own copy. Before the first lazy import in each process, materialization rechecks those links by realpath and rebinds stale links under the extension lock. This lets the registry-facing imports of a global install, npx, and source worktrees share the machine prefix while each process still gets its host’s single @cotal-ai/core registry instance; launcher artifacts are self-contained and do not resolve those mutable links later.

Reconcile policy (generation = the cotal-ai version): a never-seeded built-in is seeded; a still-installed one WE seeded (source: "seeded") is refreshed only when the version bumps (semver compare) or under --force; an operator-managed official entry (a manual ext add at a chosen version, no seeded marker) is left untouched on upgrade; a deliberately-removed one stays removed. The ever-seeded authority (seed/authority.json, mirrored to a monotonic .bak) is the sole arbiter of removed-vs-never-seeded and is unioned with its backup on read, so a truncated authority never resurrects a removal. Before writing the generation stamp, setup verifies that every (re)installed extension is recorded in the manifest, present on disk with a resolvable entry file, and at the generation version. A version-skewed payload fails loud (ext seed --repair) rather than being stamped as current. A cotal older than the store’s stamped generation refuses before writing anything, rather than stamping the store back down to its own version while refreshing nothing: run the newer cotal, or ext seed --force to rebuild the store for the version you are running. --reset is not that recovery: it discards the ever-seeded authority and resurrects deliberately-removed connectors. The refusal names a concrete cotal executable only after a bounded --version probe proves that executable is at least the store generation; otherwise it retains the generic instruction. A generation advance records the exact realpath-resolved CLI entry and an ISO timestamp in seed/stamp.json, then announces the migration after that stamp commits. An older CLI includes those fields in its refusal when present; legacy generation-only stamps stay valid and retain the shorter refusal. A CLI whose package root is the repo bin/ (a source checkout, including a suite child of bin/cotal.ts) refuses that write, stamp, and generation GC rather than migrating the operator-global store. The refusal names COTAL_SKIP_CONNECTOR_SEED=1 as the way to run other commands from a checkout, since an isolated $XDG_CONFIG_HOME alone does not lift it; COTAL_HOME does not relocate this store. Isolated in-tree seed smokes set COTAL_ALLOW_CHECKOUT_SEED=1 after pointing $XDG_CONFIG_HOME at a scratch dir. An unproven entry is refused the same way: a missing identity answer is not treated as a released install.

Crash safety. One shared advisory lock (packages/workspace/src/advisory-lock.ts: atomic hard-link publish, PID + process-start liveness, bounded wait, dead-owner reclaim) guards the whole reconcile and every cotal ext mutation; a live reconcile is waited on, not mistaken for a crash. A crash cursor is journaled before each connector mutation and cleared only at the final commit, so a SIGKILL mid-run is detected on the next boot (fail loud → ext seed --repair re-installs the interrupted connector before it clears the evidence). Seed children are authenticated (they carry the live lock’s nonce + parent PID, not a bare env flag) and record a liveness marker so a post-crash repair refuses to race an orphaned installer. ext seed --reset quarantines corrupt manifest/authority state aside and rebuilds. See cli.md ext for the operator-facing flags.