Workflow runs
Keyed submission
Section titled “Keyed submission”The new run-start-keyed command binds an envelope submission ID to one host-minted run.
run-submission-status reads that binding for the authenticated caller incarnation. Both need
fresh issued reach and a digest-verified description pinned to one manager instance and epoch.
Legacy run-start is unchanged. An unsupported keyed command never falls back to it.
The scheduler must persist the submission ID, semantic fingerprint, caller incarnation,
contract digests and instance/epoch before dispatch. The client helpers prepare that snapshot
and send one attempt at a time. They pass on only signal and deadlineMs, which a keyed
request keeps at or below 30000. They refuse a reply that names another submission, instance,
epoch or fingerprint, and that outcome stays unknown. They do not schedule retries, replace
unknown work or re-pin it to another manager. Their prepared handle is process-local and WeakMap-bound: serialized JSON
cannot be rehydrated through these helpers. Persisted scheduler recovery needs separate reviewed
integration, not a cast or an automatic fresh Describe. Keep the original work reservation
while reconciling any unknown outcome.
The binding winner may make at most three admission-create attempts in its original callback
deadline, with the same admission bytes and authority. Initial authorization precedes binding,
but fresh authority rechecks before each admission attempt can fail after binding and retain
binding-only debt. The manager refuses a start while it is stopping or already at its launch
limit before it reads or writes a binding, so that refusal writes nothing. It says nothing about
an earlier attempt under the same key, which may already be bound or running.
run-submission-status takes no launch slot, so read the outcome there. A launch that fails after
admission leaves admitted with no run, and no later call launches it. Later duplicates only
read. A binding-only result is a visible reconciliation hold, not a queue or permission to repair.
An absent admission is an observation at read time, not proof that the original callback cannot
still write. A changed caller generation or manager epoch refuses rather than adopting the run.
The receiving manager or issuing host measures the request budget on its own monotonic clock, and checks the caller’s authority expiry on the wall clock. Wall-clock changes do not extend or spend the monotonic request budget. Authority expiry remains a wall-clock check. The callback deadline a status read returns is a wall-clock record of that window, not something a caller or host enforces.
The original callback timer bounds waiting and starting more attempts. It does not cancel an already-issued broker publish, which may land later with the same frozen admission bytes. No new admission attempt starts after the request budget or authority window ends. Once an admission exists, the activation check uses only the manager’s remaining monotonic request budget; it does not recheck the caller’s wall-clock expiry. Keep the reservation even if an earlier read observed no admission.
Submission state activated reports evidence of activation, including completed and failed
runs. It is not a running-state or active-lane signal. Use native run-status for lifecycle
state. Terminal submission bindings remain permanent, and duplicate starts never relaunch them.
This implementation is under validation. Static native and signerless issuer paths require separate acceptance evidence before release or integration.
Concept (informative) · For: people writing a durable multi-agent workflow, and implementers hosting one · Normative: SPEC §14 and the language reference
spec/cotal-lang.md
A workflow run is a program that coordinates agents over hours or days and survives the
process that started it. The program is written in Cotal Lang, a small subset of JavaScript in
which every interaction with the world is one of a dozen effects (spawn, turn, ask,
checkpoint, sleep, wait, notify, monitor, and the four concurrency scopes) and everything
else is ordinary, pure JavaScript. Every effect is written into the run’s step journal before
it is performed and settled after, keyed by where in the program it happened rather than by when,
so a run that dies is resumed on any host by re-running the program from the top with recorded
effects returning their recorded results. Nothing about the interpreter is ever serialized: the
journal and the program are the whole state.
A first program
Section titled “A first program”const planner = await spawn("planner")const builder = await spawn("builder", { worktree: "wt-1" })
const plan = await ask(planner, { name: "plan", schema: { steps: "array" } })const ok = await checkpoint("approve-plan", "Approve the plan?", { timeout: "4h", onExpiry: "proceed" })if (ok.status !== "resolved") { await notify([planner], { decision: "approve-plan", outcome: "expired" })}
const r = await turn(builder, { name: "build", deadline: "30m" })if (r.status === "blocked") { await turn(planner, { name: "unblock" })}
const outcome = await race({ reply: () => wait(replied(builder), { timeout: "20m" }), giveUp: () => sleep("1h"),}, { name: "await-or-move-on" })log("outcome", outcome.index)Read it as the flowchart it is. spawn brings agents in; ask is the narrow case where the
program itself needs a value (schema is a record the program hands the handler unchanged; the
language hashes it and gives it no meaning, and the handlers in this repository enforce it as the
shorthand of the language reference §6.5);
checkpoint is a durable pause a human resolves from anywhere, raced against a durable timer; turn
wakes an agent for one turn and returns how it yielded; race runs two branches and keeps the one
whose recorded clock is earliest. Agents talk to each other in channels as they always do; the
program never speaks in a channel, and the one thing it can put in front of an agent (notify) is a
bounded decision record, not prose.
The mental model
Section titled “The mental model”- Pure code is JavaScript. Loops, records, arrays, closures, template literals, destructuring,
try/catch, arithmetic,switch, compound assignment, optional chaining, spread and rest: what you would write anyway, with the parts that hide effects or make meaning depend on the host removed (class,this,new,for...in,==, labels, regex literals,Math/Date/JSON, promises, generators). Every refusal names its code and the edit that fixes it. The builtins are a short list (keys,map,sort,json.stringify,now(),random()), and arrays, strings and numbers answer their usual methods (xs.map,s.trim(),n.toFixed()) and nothing outside that table. Records and arrays you build are yours to change until they cross an effect boundary; a member you do not own, a host prototype, or a value another branch built is refused with a code, never a surprise. - Every effect is journalled and hashed. A step is keyed
(scope path, kind, name, occurrence)and its inputs are hashed. Reorder your program, add a step, rename a variable: recorded steps still match. Change what a step asks (a checkpoint’s prompt, a sleep’s duration, a turn’s deadline) and the resume stops with a divergence naming the step, rather than replaying an answer to a question the program no longer asks. - Concurrency is visible.
parallel,race,fanOutandconclaveare the only ways to do two things at once, each branch gets its own journal namespace, and the scope writes its own entry saying how it settled: which arm won a race is a recorded fact, decided by the arms’ recorded clocks and declaration order, never by a scheduler. A failure the program itself caused inside the scope settles the entry under its own catalog code (fanOutwithout a stable key isL3021, kindruntime), so a resume reads the same code the live run threw; a plain failure from the handler records the genericL4000scope-fault. A value the program throws inside a branch is recorded with that fault, socatchreceives the same value on resume, and in a migration’s dry walk, as it did live. A branch may not write to anything declared outside it; return the value and read it out of the scope’s result. - Time and randomness are tamed.
now()is the branch’s run clock, the end of the last effect it awaited;random()is a seeded stream derived per scope. Both replay identically. - Values freeze at the boundary. What crossed into or out of an effect is what the journal recorded, and it cannot change afterwards; build a new value.
- The journal is the debugger. Every entry carries its key, its inputs’ hash, its outcome and
its timing, and every error is in the program’s own coordinates. A run can be simulated with a
scripted handler and dry-run to a plan before it touches an agent. The simulator is
discrete-event: timed effects park at their wake times and are delivered in wake order on one
virtual clock, so concurrent branches accumulate the durations they wrote and a simulated
raceis decided by the same rule a live handler produces (least recorded clock, ties by declaration order). Asleep("1m")arm beats asleep("1h")arm whatever their declaration order. Behind the worker bridge the simulator delivers its next wake only after the thread reports it has reacted to the last one, so a bridged simulation settles a race the same way, also when the dry run’s recorder wraps the simulator.
Full rules, with every code: spec/cotal-lang.md.
Continuing a run
Section titled “Continuing a run”Resume is re-execution: the driver replays the journal, the program runs from the top, recorded steps return instantly, and the first unrecorded step is performed live. It refuses a journal that belongs to another run, a pin that differs from the recorded ones, and a different language version.
A failed journal entry replays its error, while a pending entry lets the handler recover the external work it bound before the interruption.
A relative sleep completes if its duration elapses during authorized arming before a pause
is minted. It creates no past-due checkpoint. An existing pause keeps its recorded deadline and
settlement, and cancellation still rejects the sleep. Checkpoint and ask expiry rules are unchanged.
A step that writes to a far side that honors no idempotency key (posting a comment, sending a mail)
belongs in once. A resume that finds a step inside once begun and never settled does not
dispatch it again: it opens a hold, a checkpoint under a token derived from the step’s recorded
request id, whose prompt names that id. Answer it with
cotal run answer <run> <step-key> --value <json>, and the value becomes the step’s result. An
expired hold fails the step with the catchable L4027. Only ask runs inside once, so wrap just
the step that writes: a host restart while it is in flight costs a settle.
const publisher = await spawn("publisher")const res = await once(async () => { return await ask(publisher, { name: "publish", schema: { commentId: "number" } })}, { name: "publish-360" })Migrate moves a run onto edited source. A dry walk of the new program over the recorded journal
finds every recorded step the edit changed (a divergence) and every one it no longer reaches (an
orphan), and the orphan table says what each means: a removed sleep is nothing, a removed turn
already happened, a removed spawn is a live agent you must adopt or release, a removed resolved
checkpoint is a human decision you must explicitly discard. The decision is filed as a
migration record with the actor’s name on it. An adopted seat (--adopt <name>#<uid>) goes to
the edited program’s next spawn of that persona, which returns the recorded handle and mints
nothing, so the agent keeps its identity, its worktree and its turn history across the edit. A
released seat (--release <name>#<uid>) is despawned when the migration commits, through the same
discharge a cancelled branch’s seat leaves by, so the record never claims a release nothing did.
The spawn that adopts a seat binds the orphaned spawn’s goal as its own, so a resume of that step
reads the same seat back and a cancellation of it despawns the seat it holds.
Fork starts a new run from a named step of an old one, copying the prefix under the parent’s
pins (seed included, so the copied history’s pure draws are the same draws). The child is a new run
under a new id whose record names the parent and the cut step (forkedFrom); the parent is
untouched. A spawn inside the copied prefix is honoured by its onFork: "adopt" copies it, and
the child shares the parent’s agent (the manager shows that seat one turn at a time across both
runs); "respawn", the default, would mint a fresh identity the copied turns do not address, so
this host refuses that cut (L5019) rather than rewriting the parent’s history.
Fork planning and migration inspection use the recorded language version. Version-1 history uses the interpreter. Version-2 history is inspected inside a locked-down worker with a read-only journal, without a live effect handler or durable store. Inspection stops before the fork’s cut step or any effect that needs new work. Program catch and finally blocks cannot extend the cut. The recorded pins are preserved.
From an agent session
Section titled “From an agent session”Fresh cotal setup defaults declare capabilities: [spawn, run]. On an authenticated mesh,
that exposes cotal_run alongside the teammate tools. The selected manager must support execution.
Read cotal_docs pages lang-card and workflows, then try:
{ "verb": "start", "source": "await sleep(\"1s\", { name: \"first-run\" });", "file": "first-run.cotal.js"}Pass this object to cotal_run. source contains the program; file only labels diagnostics
and reads nothing from disk. The response returns a run ID before execution finishes. Call
cotal_run with verb: "status" and that runId to inspect the state and step journal.
A completed timer records its sleep step as ok.
If cotal_run is missing
Section titled “If cotal_run is missing”- Call
cotal_orientationand check the connector version, capabilities and tool list. Upgrade an older installation using the upgrade guide. - Have the operator add
runto the persona’s existingcapabilitieslist, for examplecapabilities: [spawn, run].spawnalone does not exposecotal_run. Setup leaves existing personas unchanged except for its byte-exact legacy migration. Peer persona-definition tools cannot grant capabilities. - Relaunch the agent through the manager from the updated persona so it receives newly issued
credentials and a fresh connector configuration. Editing the file or reconnecting with the
old credential does not grant new broker permissions. If the launch sets
COTAL_CAPABILITIES, update that override too; it takes precedence over the file. - Check
cotal_orientationagain, then callcotal_runwithverb: "ps"before starting work.
Tool visibility alone does not establish execution support. Hosted runs require issued caller
authority. Static authentication issues it to its credentials. On a user-auth mesh, a registered
participant manager hosts authorized runs. A managed agent invokes cotal_run through its own
launcher-provided bearer command and immutable manager instance selection. Its current managed
grant and its direct parent’s current grant must both carry run. A removed grant, stale lifecycle
or wrong manager selection refuses rather than borrowing a signed-in user’s authority.
Open meshes may expose the tool but refuse hosted runs. A local user-mode host without remote
run authority and an execution: "none" control manager also refuse the family. A legacy
credential without issued authority must be replaced through the current issuance path.
Operating a run
Section titled “Operating a run”The manager hosts runs. cotal run start hands the program to the manager of the resolved mesh
(the usual --space / --server / --creds flags), which validates it, mints the run id, drives
it in its own process, and answers with the id once the run is recorded. The terminal is free the
moment the id prints; the run continues on the manager through every pause, and a manager restart
takes back every run it had recorded running, from the journal, under the next epoch. resume
names a run the manager recorded and is refused while the manager is already driving it. ps and
journal read; answer resolves an open checkpoint, or an open ask attempt, from any terminal
or agent that holds the run capability.
journal prints an open pause’s question under its step. Once a checkpoint or ask settles with an
accepted answer, it instead prints the answer value as JSON, who answered, the artifact when one was
cited, the recorded time, and the accepted answer id. Expired pauses and ordinary steps print no
answer line.
A settled step is never answered twice, so a participant who changes their mind uses amend. It
files a new answer beside the accepted one, naming the answer it supersedes, and journal prints
each amendment under the step as an amended line, in the order the store committed them. The last
line is the current position, whatever clock each amender’s at came from. The pause stays settled
and the run keeps the answer it acted on. A step that is still open, or that settled with no answer,
refuses an amend. The manager records the amender from the credential, as for an answer, and a
spawned seat may amend only an answer recorded under its own name.
cotal run start --file build.cotal.js # the manager starts it; the minted id is printedcotal run ps # list run records: state, holder, lineagecotal run journal run-3f2a90c41b7e0d5a6c884e19b02df4a1 # print the durable step journalcotal run resume run-3f2a90c41b7e0d5a6c884e19b02df4a1 # the manager takes the run backcotal run answer run-3f2a90c41b7e0d5a6c884e19b02df4a1 "/checkpoint:approve#0" --value '"yes"'cotal run amend run-3f2a90c41b7e0d5a6c884e19b02df4a1 "/checkpoint:approve#0" --value '"no"' # record a changed positioncotal run migrate run-3f2a90c41b7e0d5a6c884e19b02df4a1 --local --file build-v2.cotal.js # check an edited program against the journalA program that does not validate is refused before anything is recorded, with every problem in the
answer as the validator would print it. The driver records the program beside the run, so resume
takes the run id alone and the manager reads the source back; an edited program is a migrate or a
fork, never a resume. cotal run migrate <runId> --local --file <program> is that check: it
replays the run’s journal and walks the edited program over it, prints whether the migration is
admissible, how many journal rows the walk accounted for, every orphaned step with its verdict and
code, and exits 0 on admissible and non-zero on not. It reads only, under the same credential
journal reads on, and the commit side is not reachable yet: the report itself says what a commit
would file and that this invocation filed nothing. An answer is recorded under the answerer the
manager knows from the caller’s credential: a managed agent by its name, anyone else by their
principal. The request
carries no name. An agent with capabilities: [run] has the same five verbs as the cotal_run
tool (MCP tools), so a program can be written and started from inside a session.
A start or resume answers once the run’s record is written, within a bounded wait; a manager
that is still taking back a predecessor’s runs at boot refuses both with unavailable, and a
retry a moment later is the whole remedy.
--local drives the run in this process instead: start, resume and answer exit when the
drive settles, --by <who> names the answerer, and cotal run resume <runId> --local --file <program> is how a run with no recorded program is continued. On a static mesh the local drive
mints the run’s own credential from the folder’s trust material, so it runs from the mesh’s
project folder. A local start also names the run’s channel ceiling itself:
--admit-read <channels> --admit-publish <channels>, comma-separated patterns or none, both
required. The record it writes says an operator admitted the run and why, and the host checks
it the same way it checks a hosted admission. A registered remote user-auth manager can host a
run through its issuing host. The host resolves a versioned caller against live issuance,
admits the run, and signs only the run’s fixed driver, mediator and one-shot operator credentials
for manager-held nkeys. Renewal checks the activated attempt; the manager holds no signer.
Local user-auth runs remain unavailable because a user bearer holds no run rows. An open mesh
hosts none either, since it issues no caller authority to admit a run under.
A logged-in user starts runs on that remote manager with cotal run start. The auth callout issues
the user’s manager connection against the user’s actor-ledger row, the CLI reads the generation
back from the connection’s accepted row, and every run verb rides the versioned rail. The issuing
host admits a run only for the owner who registered the manager, and on every resume and answer it
checks that owner and that the caller’s issuance is still live. It also watches the start, resume
and answer requests on the broker itself, and admits or issues for one the manager forwards only if
it saw that request, and only once. A resume or answer gets issuance only for the run, step, endpoint
and amendment that request named. An answer’s
credential reaches one pause, and the host reads which one off the run’s own journal, for a run
admitted on that manager’s instance. A request bound to another manager instance or epoch gets nothing, and so does one whose class or pinned contract is not the one the manager registered. Another user’s start, answer or resume is refused, and so is a revoked actor’s.
Such a run spawns, turns and despawns agents owned by that user. A spawn has the reach of the user
who started the run, as their actor-ledger row reads at that moment, and the host enrolls each agent
through its managed-agent enrollment. A spawn may be placed on the manager that hosts the run and on
no other instance. User-auth run start
records the path.
Run admission accepts canonical space names such as cotal_cloud and u_example-name
without rewriting them. Endpoint names keep their separate DNS-label grammar.
A hosted run is admitted under the caller that started it. The caller’s credential is an
issuance (identity and auth): its requests ride a
versioned rail that carries the credential’s generation, and the manager resolves that
generation’s recorded permission ceiling and writes it beside the run before the driver starts.
That ceiling, the caller’s own channel scope as it was issued, is what the run may read and post
in channels; the manager’s own reach never stands in for it. A request from a credential minted
without an issuance is refused with permission-denied and a detail naming the caller. A run
whose caller had no channels can still sleep, checkpoint and turn agents; its wait on a channel
is refused at the effect.
The manager endpoint serves run-cancel with { runId, reason }. Call it through an issued
credential on a handle pinned to the manager instance, retaining the Describe-derived epoch bind.
The reason must encode to 1 through 512 UTF-8 bytes, and the request deadline is at most 30 seconds.
There is no new CLI or MCP convenience verb in this change.
A signer-holding manager accepts the original caller’s owner, actor and lifecycle under a live
issued generation. On a signerless manager, the issuing host also accepts an interactive actor of
that owner with run, or an interactive admin. It must observe the caller’s request itself before
a manager can forward it. Managed-agent names and platform assignments do not grant this authority.
The reply separates the durable marker from the local observation: stopped means a local drive
outcome was observed; stop-requested means it was not observed within the request budget.
launch-abandoned, terminal and not-driven-here describe their respective local states.
duplicate is false for a confirmed new marker, true for a confirmed existing one, and null after
an unknown write followed by marker readback. Preserve unknown outcomes. Neither a marker nor a
reply proves process cleanup or that another host stopped. Revoked runs remain readable through
run show, but cannot receive new drive or answering authority. The endpoint’s run-status
reply carries revoked: { by, reason } independently of the recorded run state. A running record
with that marker is not proof of a stopped holder. revocationUnreadable means the marker could
not be checked; do not read that as unrevoked or resumable. A released record without a marker can
be resumed under its original admission; a released record with a marker cannot. The human
run show view displays admission revocation separately from recorded state. An unreadable marker
produces a partial result with exit 4, in human and JSON output, as it does in the run list.
A cancellation can retain a local pending intent even after a definite no-dispatch refusal if its
budget ends before the manager can confirm marker absence. That intent blocks a new local launch
or resume; it does not stop an existing drive. A later cancellation that observes a marker reconciles
the retained intent. Replaying the same request without a marker reports that it remains pending and
does not dispatch again. A manager process restart loses the local intent, but does not settle the
unknown write or prove cessation. An absent marker after an unknown write does not clear the intent
in the running process. A not-executed write outcome therefore does not promise that a local launch
is available.
Issuer observation custody has finite lifetime budgets per caller, owner and issuer. Exhaustion logs the budget class once per issuer process, without caller identities or request contents. Repeated refusals do not repeat that diagnostic. Custody entries are not evicted to make room.
Embedding hosts can call a drive’s cancel(reason) handle to signal cooperative root cancellation.
The language’s RootCancellation source carries the first reason through the walker and compiled
worker to in-flight effect contexts. It is process-local and carries no authorization. The host must
establish permission and observe the durable revocation marker before calling it. A root cancellation
ends the drive as released; the status projection can lag if its write fails. Read the recorded
state separately. A completion or failure that wins the race keeps its actual outcome. It does not stop an agent process or preempt already-pulled tool work.
release(reason) remains a separate next-effect-boundary request, used for host shutdown.
The host also supplies cancellationObserved for channel-effect classification. A fresh revocation
read together with an observed local cancellation intent ends that effect as cancelled, even while
the manager is still reading the current holder before signalling the root. Pending intents alone
never do this. An ordinary local revoke without a cancellation intent keeps its existing failure
behavior. The shared observation neither signals the root nor replaces the holder check.
If the post-marker holder read fails or exceeds the original budget, the command does not signal the root without that check and returns no stopped receipt. A parked checkpoint or sleep can remain running beside the marker. Preserve the unknown outcome. A later explicit cancellation request can read the same marker and a current holder, then report its own observed result.
cotal run revoke <runId> --local --by <who> --reason <text> writes the run’s revocation marker
from the project folder. An empty --by or --reason is refused before anything is written. The
admission itself is never rewritten. Every host reads the marker
before its next channel effect, so an open wait refuses at its next poll, and no resume,
takeover or manager restart continues the run. Revoking twice is not an error, and the first
reason stands. A run whose admission is missing or revoked is left parked by the manager’s boot
reconcile, named in its log.
run ps, hosted or --local, reads the marker beside each run record and prints revoked for a
run that carries one, whatever state the record itself holds, with the revoker and the reason
under the table. The record is display only here: a revoke writes no terminal state, because no host drove
the run to one and the journal owns the facts. A marker the listing cannot read, whether the store
is unreachable or the marker has a version or shape it does not know, prints unchecked in the
STATE column. The reason and the state the record carries go to stderr, and the command exits 1
once every row is printed. The hosted run-ps rows carry the marker as revoked (by and
reason) or a failed read as revocationUnreadable, beside the record’s own state.
A run whose step was refused (L5016) stays held; a
resume on a host that can perform the step performs it live and continues from there.
journal prints what an open pause asks beneath its step key, which is the address answer takes
back. Checkpoint expiry rides the mediated timer writer, which the delivery daemon pumps on a live
mesh; on a bare broker a pause still resolves, it just cannot expire.
Diagnostic inventory
Section titled “Diagnostic inventory”A large inventory can exceed the broker’s reply limit. A host that supports page mode can return one bounded JSON page. Page mode is not in release 0.88.0 or earlier: the calling CLI or connector and the manager both need a later version.
cotal run ps --page --on <instance> --limit 8cotal run ps --page --on <instance> --cursor <cursor>For the MCP surface, use cotal_run with verb: "ps", instance and page: { limit: 8 }.
Pass a returned nextCursor as page.cursor to continue. A managed session can use its existing
manager binding. Keep the same space, endpoint filter, instance and process epoch throughout a scan.
The CLI requires --on; page mode is not available with --local or other verbs. An older host
refuses page mode rather than returning a truncated legacy list.
Each page has at most 16 rows (default 8), and the complete reply fits 65536 bytes and the broker’s payload limit. The reader issues at most 64 storage calls and stops issuing them after 2 seconds or the smaller request duration. These are application limits, not bounds on broker CPU, disk work, client-library retries or total elapsed time. One row that cannot fit is refused, never split.
The response carries semantics: "diagnostic", the observed sequence cut, responder, storage-read
count and stopReason. Rows contain spec/status revisions and whether a revocation marker was
observed. Programs, journal contents and revocation reasons are omitted. Legacy requests without
page mode retain their existing array output.
endObserved means the walk reached its cut or received a no-match response. It does not prove an
atomic snapshot, absence, ownership, terminal work or worker inactivity. Cursors are unsigned
continuation data. Deleted history and restores can be invisible. Keep partial results incomplete,
and retain uncertain-start reservations and ownership holds. A continuation that makes no progress
is reported as incomplete; do not retry it in a loop or use it to release work.
What is on the wire
Section titled “What is on the wire”The run’s wire footprint is SPEC §14:
| Thing | Where | What it is |
|---|---|---|
| the run | run.<endpoint>.<runId> record |
the resolved pins (seed, logical epoch, budgets, language version) on the immutable half; holder, lease and journalHigh on the status half |
| the program | program.<endpoint>.<runId> record |
the source the run was started from, verbatim, written once by the driver that pinned the run; what a resume reads and what a migration is measured against |
| the step journal | WFJ_<space> stream, one subject per run |
append-only, no age eviction, no Direct Get; every append fenced by the run subject’s own sequence; takeover is replay-then-activate |
| a checkpoint answer | answer.<endpoint>.<token>.<answerId> |
the payload beside the one-use settle fact; the settle names the answer it accepted |
| a notice | notice.<endpoint>.<runId>.<addresseeId>.<noticeId> |
one bounded decision told to one agent, rendered ahead of its next turn |
| a migration | migration.<endpoint>.<runId>.<migrationId> |
the report and who applied it, keyed by the report’s own digest |
| the admission | admission.v1.<endpoint>.<runId> in cotal_admission_<space> |
the caller the run was admitted for, its channel ceiling and its provenance; written once before the driver starts, and the store refuses a second write on the key |
| a revocation | revoked.v1.<endpoint>.<runId> in the same store |
who revoked the run and why; create-only, idempotent, permanent at the broker, read by every host before its next channel effect |
The driver writes journalHigh at activation and again after each journal append, before the
program acts on the entry. A successor whose replay ends below it refuses the run with
RunJournalTailTruncated. That holds for records appended since the last activation too.
A run’s driver connects on a run-driver credential minted for one run and takeover
attempt. It can append to its journal, use its replay durable, and write its own run, program,
notice and migration records. It has no store point reads, checkpoint writes, chat consumers,
or channel and membership registry grants.
The hosting process keeps a separate run-mediator connection for effects and reads. The driver
receives methods and data from that host; it never receives the mediator credential or connection.
The host checks the journal’s current activation and step identity before dispatch, and checks
pause and wait authority again at each operation. Cancellation cleanup also admits losing steps
named by a settled parent whose cancel.issued is still false. That permission allows cleanup;
it cannot mint or rearm a cancelled pause. Wait acknowledgements consume host-held delivery
receipts. A recorded match can be reread only at its bound sequence and channel. A conclave’s
recorded ownership flag must match its step-derived channel before registry writes or cleanup.
The mediator retains endpoint-wide checkpoint rights and stream-wide leader reads as trusted host authority. Record reads exposed to the driver are restricted to its own run’s keys. Reads that decide writes remain leader-served. A read of the journal uses the run’s filtered replay durable, including the diagnostic for a journal with no run record. That durable is named after the takeover, and an attempt reads it many times, so reads under one takeover run one at a time in the hosting process and a replay removes a durable of its own name that an interrupted earlier read left behind. A durable that survives a replay’s own delete belongs to a reader the process cannot account for, and reading its tail is refused. A drive handles that refusal as a takeover does: the reads behind its steps, and the diagnostic for a journal with no run record, replay up to three times before the refusal is raised. An operator read runs under a takeover minted for that read and reports the refusal on its first read.
A served read uses a one-shot run-operator credential. An answer uses a read to find the open
pause, then a second credential pinned to that token for the answer and settlement.
cotal run --local uses the same driver/mediator split. On an authenticated mesh it needs the
locally recorded space signer to mint both credentials; a single --creds file is refused.
Direct library users supplying broker clients to MeshHandler are constructing a trusted effect
host. A hosted driver receives its closed effect interface instead.
This split confines broker credentials; it is not process isolation for injected host code.
The runtime and its effect host share the manager process. A run’s channel reach is the admitted
ceiling (SPEC §14.8): the starting caller’s issued channel scope,
recorded once in cotal_admission_<space> under a per-run run-admitter credential the driver
never holds, and re-read by the host before every channel effect. Spawn and turn keep their own
delegated checks; notify writes agent-addressed notices and is not channel publication. Treat
run as program-execution authority bounded by that ceiling, not as sandboxing of the program.
A version-1 fork can replay its settled parent history through the host. Inherited checkpoint identifiers carry no authority to read, rearm or claim the parent’s pauses. New child effects use child-derived identifiers. A fork is a new run and takes a new admission under the caller who forks it; the parent’s ceiling is not inherited.
What ships today
Section titled “What ships today”The language, its validator, interpreter, simulator and dry run are @cotal-ai/lang
(packages/lang), usable in-process with your own effect handler and with no broker: validate(src),
then run(src, { runId, handler }), and resume(src, journal, { runId, pins, handler }) to pick a
run up from its journal (the package README has the snippet, with SimHandler as the handler). That
is the in-process route, yours to drive with your own handler; a run the driver starts executes on
the compiled engine, as the engine paragraph below says. The wire
substrate of §14 (the WFJ_<space> stream, the five record kinds, the activation barrier, the
per-run grants) is in @cotal-ai/core, and the run driver, journal store, migrate and fork are
@cotal-ai/runtime (implementations/runtime). The migrate check is reachable as
cotal run migrate <runId> --local --file <program>; committing a migration it judged admissible
is not reachable from any surface yet. On the mesh handler, sleep, checkpoint,
wait(message(...)), wait(idle(...)), wait(down(...)), wait(replied(...)), notify,
spawn, conclave, ask, monitor and turn are durable.
spawn is
the manager’s spawn action submitted under the step’s own identity: the goal binds under the step’s
request id, so a resumed run re-attaches to the same seat instead of allocating a second one, a
failed or refused spawn is catchable as L4002 with the manager’s recorded reason, and a spawn on a
race branch that loses is despawned by the run’s own cancellation sweep. A seat belongs to the run
that spawned it: when the run completes, it despawns every seat it spawned, including a race
winner’s and one whose spawn failed while its process stayed up. A spawn marked onFork: "adopt"
is the exception: a fork can share that seat and no run can see whether another still uses it, so
the seat stays up until you stop it with cotal stop once every run sharing it is done. A seat a
migration handed to a later spawn follows that spawn’s policy, and it stays up if any spawn that held
it was marked onFork: "adopt", because a fork taken before the migration may still share it. In a
space with several managers, a despawn counts a seat as already gone only when the manager that
allocated it says so. A run that
fails or is released keeps its seats until a resume completes it or you stop them with
cotal stop. Start a seat with cotal spawn when it should outlive any run. permits are the budgets
this host meters: turns, how many turns the run may dispatch to the agent, and wallClock, a
duration from the spawn after which no turn is admitted. The turn that would exceed one is the
catchable L4001 (kind permit-turns or permit-wall-clock; a deadline the remaining wall clock
cannot hold counts as exceeding it), an adopted run counts the turns its journal recorded, and a
budget the host has no meter for, such as tokens or spend, is refused at the spawn rather than
accepted and ignored. supervise is the restart policy this host asks the manager to enforce:
restarts, how many in-window process deaths may come back under the same handle, and window,
the duration those deaths are counted in (default 10m). The manager restarts the process in
place under the same name, lifecycle uid, persona, worktree and permits; monitor does not fire
for a restart, and wait(down) fires only when the seat is gone for good. Spending the budget
retires the seat, and the next turn is the catchable L4002. The policy also covers the manager’s
own death. On a static mesh, when the manager ends without stopping the seat (a crash, or a pty seat
that ended with its manager), the next manager of the same instance restarts the seat the same way
and counts that as one restart. A turn the dead manager had accepted for the seat is served to the
restarted seat, so the interrupted turn runs again and finished turns do not. A seat whose budget
is spent, or whose persona file or credential changed or expired while no manager ran, is retired instead. A policy this host cannot enforce
(an unknown key, a user-mode seat, or a runtime that cannot respawn a name in place) is refused
at the spawn rather than accepted and ignored. events: false is the workflow form of
cotal spawn --no-events: the seat starts without its AG-UI event plane. A connector that
publishes none, such as Hermes, needs it, because an omitted events arms the plane and the
manager refuses that connector at the spawn. A value that is not a boolean is refused at the
spawn. conclave joins its
members to a real channel as durable membership rows: the channel derives from the step’s own
request id when the program names none (a program-named channel is borrowed, never torn down, and
a membership that predates the conclave survives its close), each member handle resolves to its
principal through the seat’s own presence row (an absent member is catchable as L4002), and a
conclave cancelled on a losing branch is released by the same cancellation sweep. ask parks one
checkpoint-plane pause per attempt, answered through cotal run answer as a checkpoint is, and
tells the agent through the same relay turn uses: one relay per attempt under the attempt’s own
token, carrying the schema, the attempt count, the deadline and the previous refusal, which the
seat’s connector renders as the record wanted and the hosted command that answers it. The relay
names the ask’s absolute deadline, so the manager stops serving an attempt at the instant its
pause expires and refuses a relay already past it. When that
literal command is run inside the managed seat, the CLI reuses the seat’s lifecycle credential and
issued caller identity rather than minting an operator instrument. The command
does not take --by: the manager records the authenticated caller as the answerer. A spawned seat’s
baseline credential carries only the self-targeted run-answer row, and the manager accepts it only
for the open ask or escalation relayed to that exact incarnation. It cannot answer another seat’s
ask, an unrelayed checkpoint, another run, or start and resume commands. An ask addresses
an agent the run spawned (anything else refuses before an attempt opens), a resumed attempt tells
the seat nothing twice, and a seat gone at the relay is L4002. On the pause itself:
the shorthand of the language reference §6.5 is enforced (an unreadable schema is L4022), a
non-conforming answer costs one attempt and its refusal reason is recorded on the entry for the
answerer to read, exhausted attempts (default one) are the catchable L4006, and so is the one
absolute deadline for the whole ask passing with no conforming record (its kind is ask-deadline),
including one that passes before an attempt’s pause is armed.
checkpoint binds what it asks on its own entry, so cotal run journal prints the question under
the step key an answer is addressed by while the pause is open: the address alone left whoever was
asked reading the source to find out what “approve” meant. The entry also records the open
attempt’s deadline and the onExpiry it was armed with, which run journal --json reports; what
an expiry does is still decided from the program’s source. An escalation’s attempt records its own
deadline, one timeout after it opens, and a resumed attempt keeps the deadline it recorded. After a checkpoint or ask accepts an
answer, the journal prints that accepted answer’s recorded value and attribution under the settled
step. A checkpoint’s comes from its frozen result, and the journal never substitutes another filed
answer or invents fields that result does not hold. An ask’s result is the value alone, so its line
is read from the answer record its last attempt’s settle named, even when that value is a record with
fields named like a checkpoint’s. Amendments print on their own lines after it.
An escalate addressed to an agent this
run spawned is relayed to that seat through the same turn relay an ask uses, carrying the prompt
and the token to answer under, held to the pause’s own deadline; a to naming anyone else is a
person, and their pause stays the one anybody can answer, with the addressee recorded and rendered
beside the question.
monitor registers interest in an agent, and the
registration is the journal entry itself, carrying the handle it registered: monitoring an agent
that is already dead succeeds, and the death is the wait’s to observe. wait(down(...)) observes
a monitored agent, and refuses one the run never performed monitor on. It reads the death off presence liveness, the
same witness a conclave join resolves members through: the value carries the handle, the reason
(lapsed when nothing live holds the name any more, superseded when a live row holds it under
a different incarnation) and the time of observation. Presence liveness skips any value without a
string card.id and card.name, so a participant that publishes a malformed row under its own key
cannot fail another run’s wait, turn or conclave join. A superseded incarnation is down at once. A
lapsed one is down only after its presence row has stayed gone for 30 seconds, because a seat whose
connector stalls past the row’s 6-second TTL, under host load or across a reconnect, renews it under
the same incarnation and is still working. The 30 seconds count only across presence reads that
each end within 6 seconds of the previous one starting, so a slow read or a run of failed reads, which
could hide a renewal, starts the count over. Even then the run asks the manager first: a seat whose
process the manager still runs is alive however long its row has been gone, so the count starts over.
Only the manager that spawned the seat can report it gone, because another manager in the space knows
only its own seats, so the run asks again until that one answers. When it does not answer, the presence
row is the only witness and the lapse stands. The asking stops at the wait’s timeout or cancellation,
and at a turn’s deadline, yield or cancellation, which then decide. A wait that begins
after the death resolves once that holds, and a timeout resolves null on one absolute deadline a
resumed run re-attaches to.
turn wakes one seat for one host turn through the manager as a pull-shaped relay: the run
submits the turn under the step’s own identity, the manager holds it as a goal pinned to the
seat’s incarnation, and the seat pulls it under its own reach ahead of its next host turn, so
nothing is pushed into a session mid-thought. The payload the seat reads names the run and the
step and carries the rendered run context, plus any pending notices addressed to it, which the
turn consumes. The seat yields through cotal_yield (done, blocked, or handoff with an
addressee), and ending its host turn yields done for every turn it was shown. A handoff names
another seat the same run spawned: the next turn in the same scope to that seat records the
link, a handoff to a name the run never spawned is the catchable L4005, and one to a seat bound
to a different worktree is L4004. The deadline elapsing before any yield is the catchable L4003:
the acceptance names the instant, the manager’s goal-bound hold denies at it, and the run arms its
own pause on that same instant, so either side outliving the other still converges on the same
answer. A seat that pulled the turn earlier stops showing it to its session at that same
instant. A seat that dies mid-turn is read off its own presence row by the run itself, with the
same 30-second confirmation for a lapsed row, and is the catchable L4002, and a death the manager
marked on the deadline terminal reads the same way. Two
turns on one seat, from two branches or from two runs, reach it one at a time: the language
dispatches the second when the first settles, and the manager shows a seat the oldest unsettled
turn alone. A turn’s goal id belongs to the caller that submitted it, so two callers may each submit
a turn under the same id. A seat yields by goal id alone, so the manager refuses to hand one seat
a second caller’s turn under an id it already holds. On an auth mesh the relay needs no extra grant: every spawned seat’s baseline
credential carries its own pull, yield, and caller-bound answer rows, the run driver’s operator instrument carries the
turn request and the inspect read that confirms a lapse, and the manager arms the deadline hold over its own serve grant and expires it
itself once due. An accept the manager cannot finish is unwound to a failed terminal on the goal
it bound, and a retry of that submission is refused naming the terminal rather than accepted a
second time.
wait(replied(...)) observes those turns from another branch: a completed turn is a reply, and
the wait resolves with the observation record (the handle, the yield’s status and note, the
yield’s own stamp). It reads as a level, the way wait(down) does: a reply that already exists
resolves the wait at once, and two replies resolve to the latest by the yield’s stamp. A denied
or cancelled turn is never a reply, so an unanswered wait rides its own mediated timeout to
null, and a handle the run never spawned or turned refuses loudly, since only this run’s turns
are observable. A turn the run itself ended without an accepted yield (its deadline, a
cancellation, a refused handoff) is never a reply, whatever the seat yields to the relay later.
A cancelled branch also withdraws what it relayed to a seat before its cancellation completes: its
turn, its ask attempt or its escalation ends cancelled through the manager’s reserved cancel
(SPEC §13.6), so the seat is not shown it after the branch’s scope settles and the next turn to
that seat does not wait behind it. Only the manager that accepted the relay holds it, so in a
space with more than one manager the run sends the cancel again while another manager refuses it,
until the accepting one answers. A cancel refused as unavailable, such as by a manager still
reconciling at boot, or one whose reply is lost (deadline-exceeded) is sent again until it
lands or the relay’s deadline passes, so the branch’s cancellation does not complete while the
seat can still be shown the relay. Any other refusal fails the branch’s step at once under its
code, since sending it again changes nothing: a manager that serves no cancel is refused
not-found before anything is sent, and an accepting manager with no such relay pending refuses
failed-precondition. The run reads the relay’s deadline from its goal and retries a failed
read, since the relay may still be served until the deadline is known. A goal record
that can never yield a deadline, such as one that is not JSON, fails the branch’s step at once,
and so does a refusal that lasts past the deadline.
A spawn may bind its agent to a logical worktree (spawn("builder", { worktree: "wt-1" })):
the handle carries the id, and the run enforces the one rule the language states about it: two
agents never share a worktree concurrently. The validator rejects the literal case up front
(L3022: two branches of one concurrent scope spawning into one literal worktree, named branch
functions included), and the runtime guards the rest, computed ids included: a spawn claims its
tree before it submits, so a second spawn into a tree held by a live seat or by a spawn still
bringing one up is the catchable L4008, a spawn that ends without a handle gives the tree back,
and the tree is reusable the moment a holder’s presence row is gone, so a discharged race loser
or a crashed seat releases its tree with no bookkeeping. A spawn the endpoint refuses at accept
is the catchable L4000 (L4001 when the refusal is the endpoint’s seat capacity), and one whose
seat never came up is L4002. A refusal that states the command did not run is answered
before it gets that far. In a space served by more than one manager the resolve and the invoke
are separate trips through the same anycast queue, so a run’s call can reach an instance it did
not resolve against, and that instance refuses ahead of any effect. The run drops its resolved
handle, re-describes and re-issues, for a bounded number of attempts; after them the refusal
surfaces as the effect’s own failure and still states that nothing ran. A spawn that names a
placement addresses one instance by name, so a refusal from it is that incarnation answering
about itself and is never re-issued. When the spawn also names cwd, that manager resolves the
existing absolute directory to its canonical host path before accepting the spawn. A missing,
relative or non-directory path refuses with no seat and never falls back to the manager workspace.
One hosted run may name one placement instance; a program naming several is refused instead of
widening one run credential across hosts. Placement is accepted only from an object-literal spawn
option bag whose endpoint and instanceId are string literals. A computed option bag, a computed
placement field, or a spread in either object refuses at run-start. The option argument may be
absent, and an object-literal bag without placement is accepted.
A turn handoff across worktrees is the L4004 described above. Recovery keeps these honest: a resumed run
reseeds its roster, holders and handoff memos from its own journal, and the driver re-issues any
recorded-but-undischarged cancellation at adoption, before the engine performs a new step, so a
loser a crash left alive does not keep its seat or its tree while the resumed run works on. The
same sweep withdraws a cancelled branch’s relays and undelivered notices: a notice waits on the run for its
addressee’s next turn, so a decision the run cancelled would otherwise arrive at an agent with
nothing to distinguish it from one that stood.
Every effect the language defines performs on the mesh handler; nothing is refused as
not-yet-durable any more. The operator surface over the driver is cotal run; the section above has the verbs.
Two engines, and which one runs your program. The tree-walker is language version 1 and the
compiled engine is version 2, two languages rather than two speeds of one (spec/cotal-lang.md
§8.4 lists what differs). The driver hosts both: every run a driver starts is stamped 2 and
executed by the compiled engine. The program runs in its own locked-down worker thread with
nothing in its global scope, while the effects and the durable journal stay in the driver’s process,
bridged over a message port. No socket or credential enters the isolate holding the program,
and every version-1 record keeps replaying on the walker, which is the walker’s job. On either
engine the driver bounds an effect’s ok result at the broker’s max_payload less 4096 bytes: a
larger result is refused ahead of the settling append (L5006), the step stays pending, and the
run is released. The driver serves a declared set of versions, and a record whose version it does
not serve is refused by name (L5023) with the run left untouched, instead of being replayed by
whichever engine happens to be present. Records do not cross between versions in either
direction; the repair is to resume on the recorded version, or to fork.
The engine needs node 22 or newer and refuses below it as EngineUnavailable, which is an
implementation limit and not a language error: it carries no L code, so there is nothing to look
up in the catalog. It is a floor rather than a warning because the engine’s frame plumbing rests on
AsyncLocalStorage, and 22 is the lowest node it has been measured on. The walker has no such floor.