Skip to main content

Agentic Development Workflow (ADW)

All feature development on Orbit is driven by the Agentic Development Workflow (ADW) — a structured pipeline where AI agents handle drafting, implementation, and automated testing, while humans stay in control of spec approval and production sign-off.

A narrated 2-minute replay of a real ADW run (Spec 99), regenerated on demand.

🛸
Interactive deep-dive

Explore the full ADW architecture hands-on — trigger a simulated agentic loop, browse the live lat.md knowledge graph, walk the sandbox lifecycle, and drive the orbit CLI in a terminal simulator. Open the deep-dive (opens full-screen).

📽
Executive briefing deck

An 11-slide interactive briefing on ADW governance and adoption, for leadership audiences. Open the briefing (opens full-screen; arrow keys to navigate).

The ADW Spec Lifecycle: From Draft to Production

Why ADW?​

Instead of ad-hoc commits and one-off deploys, every change to Orbit follows a spec-first, environment-gated loop:

  • No guessing. Every feature starts as an approved HTML spec — the agent builds exactly what was specified.
  • Sandboxed by default. All AI agent work happens in the sandbox environment. Production is never touched by an agent.
  • Evidence required. Playwright screenshots are mandatory artifacts before anything can be promoted to production.
  • Human at the gate. A human reviews sandbox output and signs off before the production gate opens.

The Five Commands​

Orbit ADW Command Reference Card
StepCommandWhat it does
1/spec-create <description>Drafts a new HTML spec in specs/, creates a spec/NN-slug branch, commits and pushes. The spec number is assigned automatically.
2/spec-reviewSpawns an AI Architect agent to critique the spec draft for technical completeness and quality before any code is written.
3/spec-implementTriggers an AI executor agent to build the approved spec within the sandbox environment, on the feature branch.
4/spec-validateExecutes Playwright browser tests against the live sandbox URL and captures mandatory evidence screenshots of the UI.
5/spec-dispatchConvenience command that runs the full end-to-end loop: sandbox reset → implement → validate. Use this for a clean run from scratch.

/spec-promote is the final step — it triggers the human sign-off flow to merge validated code into the production branch and kick off the production deploy pipeline.

Environment Ownership​

Each environment has a designated owner. AI agents never touch production directly.

EnvironmentOwnerPurpose
DevHuman DevelopersFeature branches and manual exploratory work
SandboxAI AgentsFull-stack agentic pipeline output and human preview
ProductionCI/CD (human-approved)Client-facing code; requires human-approved sign-off

The sandbox backend (orbit-backend-sandbox) is the target for all agent-driven deploys. The sandbox database (orbit_sandbox) is seeded with a deterministic 15–20 work order dataset via /sandbox-reset — this keeps Playwright validation reproducible across runs.

The Spec Lifecycle in Detail​

1. Draft (/spec-create)​

Running /spec-create "add compliance upload endpoint" in Claude Code:

  • Generates a self-contained HTML spec file in specs/ (e.g. specs/07-compliance-uploads.html)
  • Assigns the next sequential spec number automatically
  • Creates a spec/07-compliance-uploads feature branch
  • Commits and pushes the spec HTML to that branch

As soon as the spec is approved, merge the spec file to main immediately — before implementation begins. Approved specs must be readable on main; they should never be hidden behind an open feature branch.

2. Architect Review (/spec-review)​

An AI Architect agent reads the spec and checks for:

  • Missing acceptance criteria or ambiguous requirements
  • Architectural conflicts with the existing codebase
  • Missing edge cases (state gate conditions, error paths, persona-specific behavior)

The agent returns structured feedback. The spec author addresses any blockers before dispatch.

3. Implement (/spec-implement)​

The executor agent reads the approved spec and builds it:

  • All code changes land on the spec/NN-slug branch
  • The GitHub Actions pipeline triggers on push: builds the Docker image → pushes to ACR → deploys to orbit-backend-sandbox
  • A health check gate confirms the sandbox is up before the agent proceeds

The agent works entirely in the sandbox. It has no credentials or access to orbit_prod.

Quality gates (before the commit)​

Once the agent has written the code and it's been formatted and linted, the ADW build runs two automated quality gates in order, before the implementation is committed. Nothing that fails these gates becomes a commit or a PR.

  1. Ontology constraint gate (Step 6.6) — a deterministic, closed-world validator (ops ontology check) runs over the portfolio knowledge graph and enforces two hard invariants:

    • C1 · alias-uniqueness — no project id or alias is claimed by more than one project.
    • C3 · edge-referential integrity — every DEPENDS_ON / FEEDS_INTO reference resolves to a registered project, so a typo'd, renamed, or removed project reference is caught.

    A blocking violation escalates the work item to needs_human and is never committed or turned into a PR. In the ADW loop this gate is guarded by the ADW_ONTOLOGY_CHECK environment variable, which defaults to OFF (a dark launch) — mirroring the maker-checker's own rollout flag.

  2. Maker-checker verifier (Step 6.7) — the existing independent Opus adversarial reviewer, which reads the spec, the actual diff, and the test evidence and refuses to let unverified work reach a PR.

Neuro-symbolic pairing

The ontology gate is the deterministic "symbolic" guardrail that complements the probabilistic LLM maker-checker — "Pydantic at the door, ontology at the ledger." It deterministically catches a class of "impossible-in-our-domain" errors — dangling references and duplicate aliases — that are hard to catch by prompt or LLM review alone.

The same ontology check also runs as a blocking CI gate on every pull request. There it is not env-gated — it is always on — so a violation can't merge to main even outside the ADW loop.

Both gates record their catches to a metrics surface (the ontology_violations table plus an ontology_catch flag alongside the maker-checker metrics), so their effectiveness is measured over time and surfaces in the Command Center "Ontology Guardrail" Governance tab.

4. Validate (/spec-validate)​

Playwright MCP runs browser tests against the live sandbox URL:

  • Navigates the golden path for each affected flow
  • Captures screenshots at each key state as evidence artifacts
  • Reports pass/fail per test case
Evidence Rule

Screenshots are required evidence artifacts before production promotion. /spec-promote will not open the gate without validated Playwright output on file.

5. Promote (/spec-promote)​

Once sandbox output is reviewed and approved by a human:

  • The feature branch is merged to main
  • The GitHub Actions production pipeline triggers: builds → pushes to ACR → deploys to orbit-backend-prod
  • The spec is marked validated in the roadmap

Keeping the knowledge graph in sync​

The ADW loop only works because agents ground themselves in the repo's lat.md/ knowledge graph before writing code — running lat search to read documented design intent instead of guessing from source. That grounding is worthless if the graph has drifted from the code. Two gates keep them in sync, mirroring the same soft-reminder-plus-hard-gate shape as the ontology guardrail above.

GateWhenWhat it does
Soft — in-sessionevery agent turnThe lat hook injects "search the graph first" on UserPromptSubmit and "sync lat.md, then run lat check" on Stop — a reminder while the task context is still warm. A nudge, not a block. Reinforced by the CLAUDE.md post-task checklist.
Hard — at the PRCI, every PRlat check runs as a required status check. It fails if any [[wiki link]] or @lat: code reference no longer resolves, so a PR that renames or deletes a documented symbol can't merge until lat.md is updated. Offline and deterministic — no API key required.
Two gates, not one

The in-session reminder absorbs most drift cheaply; the required CI check catches the rest un-bypassably. Neither alone holds — reminders-only lets docs rot silently, CI-only turns every PR into a fight. Together they form a ratchet that only turns toward documented. It's the same pattern as the ontology gate: a deterministic, offline validator wired as a required check.

Two commands surface the loop: ops lat check runs the gate locally with the same result as CI, and ops lat gates reports which repos have the hard gate wired — enforced, partial, or missing.

Branch Naming Convention​

All feature branches follow the pattern spec/NN-slug:

spec/07-compliance-uploads
spec/08-hauler-notifications
spec/09-cietrade-sync

The number matches the spec file number. /spec-create handles this automatically — never manually create a branch for ADW work.

Structured Roadmaps, Spec Drafting & CI Enforcement​

Orbit tracks specs through a hand-maintained HTML roadmap (see Development Roadmap). Newer ADW projects — starting with Avalara Commit & Reconciler — run an evolved, machine-enforced variant of the same loop, where the roadmap is a structured data file that AI and CI write deterministically. If you are standing up a new ADW project, use this pattern.

Drafting specs — use OpenSpec​

Draft specs with OpenSpec, a lightweight spec-driven-development framework: you and the AI agree on what to build before any code is written. Work is modeled as a change (openspec/changes/<name>/) built from four artifacts in order — proposal.md → design.md → capability specs/ → tasks.md — driven by /opsx:* slash commands (/opsx:continue, /opsx:update, /opsx:apply, /opsx:verify, /opsx:archive) plus the openspec CLI (openspec status | validate | list).

Specs use a structured requirement syntax rather than prose — ### Requirement: … SHALL … with #### Scenario: blocks (WHEN / THEN / AND) — which is what makes each requirement independently testable.

Why OpenSpec (our recommendation)

It keeps requirements out of chat history and in the repo as the source of truth, it's brownfield-first (changes are deltas against a baseline), and it maps cleanly onto the roadmap below — one roadmap deliverable ↔ one or more OpenSpec capabilities. Recommend it as the drafting standard for new ADW projects; there's no need to hand-generate a second, parallel spec set.

The roadmap is a structured TOML file​

Each project owns a single roadmap/roadmap.toml — the authoritative roadmap. roadmap.html and any dashboard payload are generated from it and never hand-edited. Every deliverable carries a forward-only lifecycle enum:

planned → implemented → validated → deployed

plus the OpenSpec capabilities it maps to, its dependencies, and a [deliverable.validation] block declaring the artifact that proves it (a Playwright walkthrough for UI, a pytest RESULTS.md for backend, a sandbox reconciliation report for config).

A zero-LLM tool, scripts/roadmap.py, does the work:

CommandWhat it does
roadmap.py statusTable of every deliverable + its lifecycle state
roadmap.py syncAdvances status from evidence — merged spec PRs → implemented; validation artifact on disk → validated
roadmap.py renderRegenerates roadmap.html from the TOML
roadmap.py lintValidates the TOML and enforces the gates (this is what CI runs)

TOML sync — exactly when status changes​

The status bump rides inside the spec's own PR (atomic), so the roadmap is correct the moment the PR merges:

  • planned → implemented — the spec's PR merges to main.
  • implemented → validated — the declared validation artifact exists on disk. A deliverable cannot reach validated without it.
  • validated → deployed — the deploy pipeline succeeds.

A post-merge (and weekly) roadmap.py sync runs only as a drift safety-net. Because status is a projection of structured data, there is no LLM reconciliation, no bot identity, and no admin-PAT auto-merge — unlike the Orbit HTML roadmap, which is reconciled after the fact.

CI gate enforcement​

A branch-policy build validation runs roadmap.py lint on every PR into main. It fails the PR when:

  • roadmap.toml is invalid or a dependency doesn't resolve,
  • a deliverable is marked validated without its declared validation artifact on disk,
  • roadmap.html is stale (regenerate with roadmap.py render).

This is what makes "done" mean proven: you cannot merge a spec that claims validation it can't show — the same spirit as Orbit's Evidence Rule, now enforced by CI rather than convention. The gate is deterministic Python — no LLM, ~$0. Application Test/Build/Deploy CI is separate and standard, added with the project's scaffolding.

Cost note

Projects can run ADW locally, on a developer's Claude Code subscription rather than the metered pipeline. The roadmap tooling and its CI gate are pure Python, so they add no LLM cost either way.

Supporting Commands​

CommandPurpose
/sandbox-resetTear down and reprovision sandbox to a clean, seeded state. Run before a fresh /spec-validate cycle.
/deploy-sandboxDeploy the current branch to sandbox without going through the full spec flow. Useful for hotfixes or manual testing.
/deploy-prodProduction deploy checklist and instructions. Shows what will be deployed and prompts for sign-off confirmation.

Which entry point should I use?​

There are three ways to kick off an ADW run, ordered from most hands-off to most manual. Pick based on how much setup you have locally and how much control you want.

Entry pointBest forWhat you getPrerequisites
A · GitHub issue + adw labelMost colleagues; zero local setupOpen an issue, apply the adw label, and CI (adw-issue-trigger.yml) authors the spec, self-reviews, implements, and opens a draft PR automatically.Just a GitHub account with write access. The CI does everything; a human still runs /spec-validate and reviews the sandbox before merge.
B · /adw <request> in Claude CodeDevelopers who want the full-powered pipelineThe complete local loop: 7-dimension architect review with revision loops, executor implementation, and Playwright validation with video — capabilities the CI runner can't provide.Run from the mars_workorder_system directory locally. Needs gh + az authenticated, ~/.sling/.env, Node/Python/uv, the Playwright MCP server (--caps=devtools), oh-my-claudecode, and the lat CLI.
C · Individual stage commandsManual control, retries, or debugging a single stageRun any one stage on demand: /spec-create, /spec-review, /spec-implement, /spec-validate, /spec-promote, /spec-dispatch, /sandbox-reset.Same local prerequisites as B. You manage the spec's Approved badge and stage ordering yourself.

Rule of thumb: if you just want a feature built and don't have the local toolchain, use A. If you're a developer who wants the highest-quality run with architect review and Playwright evidence, use B. Reach for C only when you need to re-run or debug a specific stage.

Top gotchas​

The five things that trip up new users most often:

  1. PR creation is the sandbox deploy trigger. There's no push-to-spec/* trigger — backend-deploy.yml and azure-static-web-app.yml both fire on pull_request → main. Push a branch without opening a PR and the sandbox never deploys; /spec-validate then fails with "No SWA preview URL found." (/spec-implement opens the PR for you.)
  2. The SWA preview URL lives in the Azure bot's PR comment — not in gh pr view --json url (that's just the GitHub PR page). The Azure bot posts the live frontend preview as a comment in the form https://<hash>-<pr#>.<region>.azurestaticapps.net. Watch out for the eastus2 region segment tripping up naive extraction scripts.
  3. Health checks must be content-aware. The Azure Container Apps "Welcome" placeholder page returns HTTP 200 for any path, so a status-code-only check passes even when the backend is down. Every health check in the pipeline asserts a JSON status field in the response body — copy that pattern for any new check.
  4. The spec must carry the Approved badge before /spec-implement or /spec-dispatch will run. A spec stuck at Draft or Review is rejected. /adw handles this via its Stage 2 review gate, but when running stages manually you must update the badge in the spec HTML yourself first.
  5. The loop deploys code, not infrastructure. backend-deploy.yml ships containers and runs migrations — it does not run Bicep. Any spec that provisions a new Azure resource needs a manual az deployment group create step first; validation will fail until that resource exists.

See Also​