Skip to main content

G. Migration Frameworks & Roadmap

Wasteology is at a decision point, not yet past one. Two cloud migrations — GCP→Azure and GitHub→ADO — are plausible but not inevitable, and the cost of a wrong decision in either direction outweighs the cost of taking a few extra weeks to decide well. This section frames each migration as a go/no-go decision first, then provides a phased plan conditional on a "go" verdict, and closes with a unified rollout roadmap that sequences all standards from the other sections plus a concrete 90-day plan for the incoming Product Owner.


G1. GCP→Azure Migration Decision Framework

What lives on GCP today

Wasteology operates two tightly coupled applications on GCP, both in GCP project academic-torch-405913 (us-central1):

SystemGCP ResourcesWhat It Does
Invoice Listener2 Cloud Run Services (wasteololgy-email-handlers, wasteololgy-onedrive-handlers), 3 Cloud Run Jobs (email/OneDrive/error-retry schedules: 10 min / 1 hr / 1 hr), 1 VPC + Cloud NAT, Artifact Registry, BigQuery datasetPolls email and OneDrive for inbound invoice PDFs; OCRs them via Document AI; publishes results to Pub/Sub and GCS
Invoice Management (Trashy)2 Cloud Run Services (FastAPI backend + Nginx/React frontend), 3 Cloud Functions (doc_ai_processor → db_inserter → webservice_caller Pub/Sub chain), Cloud SQL PostgreSQL (trashy schema, 283,725+ invoice rows), 3 GCS bucketsFull-stack invoice management app — receives results from Listener, stores PDFs and extracted data, provides management UI
Document AIProcessor 9e41876640558aee ("REPUBLIC Invoice Parser")GCP-native OCR + entity extraction — no direct Azure equivalent at the same price point
Terraform IaCState in GCS (wasteology-terraform), separate prod/dev configs with modules (sql, storage, document_ai_pubsub)Infra-as-code; prod is mature with HA Cloud SQL and deletion protection ON

Key external facts:

  • Already has Azure dependencies: secrets reference Azure SQL (wasteology.database.windows.net), Azure Blob (pdfinvoices.blob.core.windows.net), Microsoft Graph, Entra ID. GCP is not isolated — it is already a hybrid.
  • Document AI has no direct Azure Form Recognizer parity for invoice parsing without retraining.
  • IaC is Terraform (GCP-native), not Bicep.
  • Production deployments are still manual (deploy.sh); CI/CD triggers exist only in dev.
  • ABS is the external Azure admin partner — any Azure landing zone work requires their involvement.

Go/No-Go Criteria

Evaluate each criterion and record the finding. A preponderance of "Against" findings is a no-go.

CriterionFor MigrationAgainst MigrationWasteology-Specific Finding
Cost — computeAzure Container Apps (ACA) pricing at Wasteology's scale may be similarCloud Run scales to zero; ACA consumption pricing also scales to zero — washMonitor actual GCP bill for 30 days before deciding; if < $200/mo the cost case is weak
Cost — data transferAzure Blob already holds legacy PDFs; eliminating GCP→Azure egress saves moneyGCS→ACA data transfer pricing must be modeled, not assumedRun gcloud billing export for 3 months to quantify egress cost
Document AIAzure AI Document Intelligence handles invoicesRetraining cost + model quality delta unknown; REPUBLIC Invoice Parser is production-tunedRequires a proof-of-concept extraction run on 100 real invoices before go
Data gravityProduction data (PostgreSQL trashy schema) could move to Wasteology's existing Azure PostgreSQLCloud SQL → Azure Flexible Server migration is well-tooled (pg_dump + pgcopy)One-time migration; ongoing gravity is neutral post-cutover
Team skillsTeam already operates Azure dailyNo one currently manages GCP infra day-to-day; KloudStax set it up; runbooks are thinStrong for — GCP is orphaned knowledge; Azure is active team capability
Vendor lock-in (GCP)Removing GCP eliminates a second cloud bill, a second IAM model, a second monitoring stackStrong for — single-cloud simplicity is worth significant one-time migration cost
Vendor lock-in (Document AI)Replacing a GCP-specific ML service with Azure equivalent is the riskiest stepThis is the migration's gating risk item
Operational burden of split cloudTwo clouds = two incident on-call stacks, two cost monitors, two identity planesStrong for — team is three people; split-cloud ops is punitive at this scale
CI/CD maturity on GCPProd deployments are still manual; migrating to Azure means rebuilding CI/CD once rather than twiceMigration is an opportunity to close the CI/CD gap in one pass
Contractor continuityKloudStax built the GCP stack; if they are off the engagement, tribal knowledge is goneStrong for migration before that knowledge exits further
PO decision point

After reviewing the 30-day GCP bill and the Document AI PoC results, the PO and dev lead jointly record a go/no-go verdict in docs/decisions/adr-001-gcp-azure.md. If no-go: document the criteria that blocked it and revisit at the next half-year review.


If Go: GCP→Azure Migration Phases

PhaseNameActivitiesExit CriteriaRough Duration
G-0AssessAudit GCP bill (3 months); PoC Azure AI Document Intelligence on 100 real invoices; map all secrets from GCP Secret Manager to Azure Key Vault equivalents; inventory Terraform modules to Bicep equivalentsPoC extraction quality ≥ current Document AI; cost model approved3–4 weeks
G-1Landing ZoneProvision ACA environment in Wasteology Production sub; create ACR repository for invoice images; migrate secrets to KV; set up ADO pipeline scaffolding for invoice listener + management builds; verify Entra ID auth still works from ACAADO pipeline builds and pushes a test image successfully2–3 weeks
G-2Migrate ListenerPort Cloud Run Jobs → ACA Jobs with KEDA or timer triggers; port Cloud Run Services (email/OneDrive handlers) → ACA; replace Pub/Sub with Azure Service Bus (or Azure Storage Queue — already referenced in handler secrets); replace Document AI calls with Azure AI Document IntelligenceListener processes a real test invoice end-to-end in ACA dev environment4–6 weeks
G-3Migrate Management**Port FastAPI backend + React frontend → ACA; replace Cloud SQL with Azure Flexible Server (PostgreSQL); migrate trashy schema data via pg_dump/restore; replace GCS PDF buckets with Azure Blob (already partially in use); port Cloud Functions Pub/Sub chain → Azure Functions or inline ACA logicInvoice management UI loads in ACA; all CRUD operations validated; zero data loss verified4–6 weeks
G-4CutoverDual-run for 2 weeks (GCP + Azure both receiving traffic); validate invoice counts match; migrate remaining Terraform state to ARM/Bicep or leave Terraform for GCP teardown; update external DNS/URLsInvoice counts identical between systems for 5 consecutive business days2 weeks
G-5DecommissionDestroy GCP Cloud Run, Cloud Functions, Cloud SQL, Pub/Sub, Document AI processor, Artifact Registry; cancel GCP project billing; archive Terraform configs in ADOGCP bill reaches $0; no active GCP resources1 week

Preserve through migration: all PDF history in Azure Blob (already partially there); the existing Azure AD auth integration; invoice numbering continuity; the trashy PostgreSQL schema structure.


G2. GitHub→ADO Migration Decision Framework

What lives on GitHub today

Core repos are already ADO-only. Three repos remain GitHub-primary:

RepoGitHub StatusWhy It Matters
wasteology/mars_workorder_systemGitHub-primary — GitHub Actions OIDC deploys backend to Azure Container Apps (Wasteology-Orbit sub); merge to main auto-deploys to productionMoving means rebuilding OIDC trust, GitHub Actions workflows → ADO Pipelines, and branch protection rules
wasteology/mars_workorder_system (prefect-azure-infrastructure)GitHub-primary — trunk-to-main triggers ADO pipeline for ETL repo CI, but the infra repo itself lives on GitHubHybrid already: GitHub repo triggers ADO pipelines via cross-repo service hooks
dbt_wasteology_warehouseGitHub-primary, ADO-archival — ADO mirror exists (d1e5763a-fcbf-4af3-bfed-a8fbaf04788c) but GitHub is canonical; dbt Cloud CLI auto-connects via GitHub integrationdbt Cloud requires GitHub or ADO connection; switching changes the dbt Cloud project settings

Key facts:

  • ADO org wasteology (single project) already hosts all operational repos; ADO bills to Wasteology Production sub a5a6818e.
  • Mars/Orbit uses GitHub Actions OIDC — a direct trust between GitHub Actions runners and the Wasteology-Orbit Azure subscription managed identity. Migrating means re-establishing that trust via ADO Workload Identity Federation instead.
  • prefect-azure-infrastructure is a hybrid: repo on GitHub but CI pipelines run in ADO. This is already a maintenance tax.
  • dbt Cloud CLI has first-class ADO integration; this is not a blocker.

Go/No-Go Criteria

CriterionFor MigrationAgainst MigrationWasteology-Specific Finding
ConsolidationSingle SCM, single PR workflow, single access control model, single audit logStrong for — two SCMs for a three-person team is a cognitive and security surface tax
Working CI/CDADO Pipelines already proven for all core reposGitHub Actions OIDC for Mars is working; rebuilding it is effort with no user-visible valueThe rebuild is one-time; the ongoing cost of split SCM is permanent
GitHub Actions vs ADO PipelinesADO Pipelines supports YAML, multi-stage, approvals, environments, OIDCGitHub Actions has broader marketplace; specific actions (e.g., Docker buildx, dbt Cloud CLI action) need ADO equivalentsAll actions used are available as ADO tasks or inline script steps
Auto-deploy wiring (Mars)ADO service hooks can trigger pipelines on push to mainGitHub merge→deploy is a two-line workflow; ADO equivalent requires a pipeline + service hookMars go-live is Aug 15, 2026 — migration timing matters
dbt Cloud integrationdbt Cloud supports ADO nativelySwitching repo provider in dbt Cloud requires re-running project setup wizardLow-risk; 30-minute task; ADO integration is fully supported
Branch protectionADO branch policies (required reviewers, build validation) cover all GitHub branch protection featuresFeature-equivalent; migrate policies 1-to-1
Secrets / OIDC trustADO Workload Identity Federation is the Wasteology standard (already used for wg-azure-infra)OIDC trust must be re-established for each Azure subscriptionOrbit sub OIDC setup follows the same pattern already in wg-azure-infra
Contractor accessADO access provisioning via the established onboarding runbookGitHub org membership is a separate entitlement planeFor — one identity plane, one off-boarding step
PO decision point

The recommended default is go for GitHub→ADO given that the long-run consolidation benefit is clear and no technical blocker exists. The only timing question is whether to migrate mars before or after the Aug 15 go-live. Recommended: migrate mars after go-live to avoid introducing churn into a live launch. Lock the decision by Day 30.


If Go: GitHub→ADO Migration Waves

WaveReposKey ActivitiesPreserveTiming
Wave 1prefect-azure-infrastructureMove repo to ADO; update the ADO service hook that already triggers this repo's pipelines; no CI/CD rebuild needed since pipelines already run in ADOExisting ADO pipeline triggers; branch namesAfter PO decision (Day 30+)
Wave 2dbt_wasteology_warehousePromote ADO mirror to primary; update dbt Cloud project settings to point at ADO; archive GitHub repodbt Cloud job schedules; existing branch history via git push --mirrorAlongside Wave 1 or immediately after
Wave 3mars_workorder_systemCreate ADO repo; set up ADO Workload Identity Federation trust with Wasteology-Orbit sub; port GitHub Actions workflows to ADO YAML pipelines; configure branch policies (1-review minimum, build validation); update OIDC trust in Azure; test deploy-to-prod pipeline end-to-end; archive GitHub repoOIDC trust → recreate as ADO WIF; branch protection → ADO branch policies; GitHub environment secrets → ADO pipeline variables backed by KVAfter Mars go-live (post Aug 15)

For all waves: export full git history (git clone --mirror); push to ADO with git push --mirror; update any CI badge URLs in docs; update the project manifest TOML in wg-orchestration/manifests/builtin/ to reflect the new ado_repo field.


G3. Unified Phased Rollout Roadmap

This roadmap assumes the other sections' standards (ADO/repo, dev-env/CLI, docs, agentic workflows, access governance, contractor onboarding) are ratified by the PO in Phase 0. Dependencies flow left-to-right: later phases cannot start until their predecessors' exit criteria are met.

PhaseNameTimingSections ActivatedKey DeliverablesExit Criteria
Phase 0Foundation & RatificationWeeks 1–4 (Days 1–30)All sectionsPO ratifies standards doc; ADO repo + branch-policy template created; dev environment bootstrap script tested on one new machine; lat check passes on all owned repos; access governance RACI signed; contractor NDA + access-request runbook publishedZero TODOs in the standards doc; every owned repo passes lat check; RACI has named owners
Phase 1Dev Environment & Repo NormalizationWeeks 5–10 (Days 31–70)B (dev-env), A (ADO/repo)All owned repos adopt uv, ruff, pytest, .python-version; ADO branch policies (1-review, build validation) applied to all owned repos; CLAUDE.md + lat.md present in every repo; ops doctor passes on every dev machineuv sync && uv run pytest green on all owned repos; lat check clean across portfolio
Phase 2Docs & Agentic WorkflowsWeeks 9–14 (Days 55–100)C (docs), D (agentic)Internal docs site sections up-to-date for all owned projects; docs-triggers.yaml present in each repo; ADW pipeline running for at least 2 repos; global agents deployed to wg-agent-infra; session-wrap / prime hooks activeDocs site auto-update fires on a real code change; ADW ships at least one real feature
Phase 3GitHub→ADO ConsolidationWeeks 11–18 (Days 75–125)A (repo)Waves 1–2 (prefect-infra + dbt warehouse) migrated; Wave 3 (mars) planned and scheduled post-Aug-15 go-liveWaves 1–2 live in ADO; GitHub repos archived; no new commits to GitHub for migrated repos
Phase 4Access Governance HardeningWeeks 13–20 (Days 90–140)F (access governance)Azure RBAC cleanup: Reader + Cost Management Reader on all 3 subs granted to wg-azure-infra-identity; Graph Application.Read.All granted; daily snapshot capture no longer partial; drift detection firing and actionedops azure snapshot captures complete (no partial-capture guard fires); digest AZURE GOVERNANCE block shows real cost data
Phase 5GCP Decision & Migration (conditional)Weeks 17–40+ (Days 120–280)G (this section)Go/no-go verdict in adr-001-gcp-azure.md; if go: phases G-0 through G-5 executed; if no-go: GCP tagged as legacy with an annual review dateIf go: GCP bill at $0; Trashy on ACA. If no-go: ADR filed and dated
Phase 6Contractor Onboarding & Steady StateWeeks 20+ (Days 140+)E (contractor onboarding)Onboarding guide live on internal docs site; first external contractor onboarded via the standard runbook; time-to-productivity measured; feedback loop back to Phase 0 standardsFirst contractor reaches "ready to commit" in ≤ 1 business day; time-to-onboard logged

Cross-phase dependencies:

  • Phase 1 must complete before Phase 2 (docs tooling requires lat check to be green).
  • Phase 3 Wave 3 (mars) is hard-gated on Aug 15 go-live; do not let migration risk a live launch.
  • Phase 4 unblocks full dashboard cost visibility; schedule alongside Phase 3, not after.
  • Phase 5 (GCP) is independent of all other phases and can run in parallel with 3–4 once the go/no-go is recorded.

G4. The Incoming Product Owner's First 90 Days

Days 1–30: Orient & Ratify

The first month is about understanding what exists, committing to decisions that unblock the team, and establishing the cadence that will persist for the next year. Avoid making structural changes before understanding the system.

WeekActionTool / Artifact
Week 1Shadow one full ops digest daily cycle end-to-end — from morning brief to wdp-palantiri dashboarduv run ops digest daily; wdp-palantiri
Week 1Read all lat.md sections for the 5 owned projectslat search "overview" from each project root
Week 1Review the Azure subscription map; confirm ABS contact and escalation pathlat section cloud-governance#Subscription Map; intro call with ABS
Week 1–2Review open ADO work items; identify the top 3 blocking items the team is waiting on a decision forADO Boards — wasteology org
Week 2Ratify (or formally amend) the standardization document — sections A through Gdocs/strategy/standardization/
Week 2PO decision point — GitHub→ADO: record go/no-go verdict in docs/decisions/adr-002-github-ado.mdTemplate: date, options considered, decision, rationale, next review
Week 3Commission the 30-day GCP bill review and Document AI PoCAssign to dev lead; set a Week 6 due date
Week 3Confirm Mars go-live plan and freeze code/infra changes in the two weeks priorCoordinate with Mars dev team
Week 4Establish weekly cadence: Monday morning digest review, monthly RBAC review, quarterly SP cleanupBlock calendar; assign owners per RACI below
Week 4Review the Portal Reconciliation Phase 4 deploy (annotation UI) — it is merged but not deployedops recon CLI; wdp-palantiri page 11

Days 31–60: Pilot & Validate

The second month runs the first wave of standards in practice and evaluates the GCP decision.

WeekActionTool / Artifact
Week 5–6Dev environment bootstrap: new developer (or contractor) onboards using the Phase 1 standard; PO observes and measures timeSection B standard; uv sync && uv run pytest
Week 5Apply ADO branch policies to one repo as pilotADO → Repos → Branches → Branch Policies
Week 6GCP cost + Document AI PoC results due; record PO decision point — GCP→Azure in docs/decisions/adr-001-gcp-azure.mdGCP billing export; Azure AI Studio PoC
Week 6–7GitHub→ADO Wave 1 (prefect-infra) if go decision was recorded in Week 2git clone --mirror; ADO repo create
Week 7Deploy Portal Reconciliation Phase 4 annotation UI to wdp-palantiri./scripts/build-and-push.sh --deploy; validate page 11
Week 7–8GitHub→ADO Wave 2 (dbt warehouse); update dbt Cloud project settingsdbt Cloud project settings; ADO mirror promotion
Week 8Run lat check across all owned repos; remediate any broken links or missing sectionslat check in each repo
Week 8Measure Phase 1 exit criteria: uv sync && pytest green, branch policies live, CLAUDE.md presentAutomated check in ops doctor

Days 61–90: Measure & Harden

The third month locks in Phase 2–3 progress, records baseline metrics, and sets the cadence for steady state.

WeekActionTool / Artifact
Week 9–10Docs site: verify all owned-project sections are current; run docs-triggers.yaml end-to-end test./scripts/deploy.sh internal; internal docs site
Week 9ADW pipeline: verify at least 2 repos are using the ADW workflow for feature/bug/chore ticketsADO Boards; ops digest daily ADW section
Week 10Grant wg-azure-infra-identity MI the remaining RBAC (Reader + Cost Mgmt Reader × 3 subs, Graph Application.Read.All)ABS-assisted; az role assignment create
Week 10–11Verify ops azure snapshot captures full data (no partial-capture guard); review digest AZURE GOVERNANCE blockuv run ops azure snapshot; morning digest
Week 11Mars go-live (Aug 15) — post-launch stabilization before migration beginsCoordinate with Mars team
Week 12Plan GitHub→ADO Wave 3 (mars) — schedule for post-go-live, target Week 14–16ADO repo setup; OIDC WIF plan
Week 12Record 90-day metric baselines (see G5)Metrics spreadsheet or ADO dashboard
Week 13Retrospective with dev team: what friction remains; amend standards if neededUpdate docs/strategy/standardization/

G5. Governance: RACI & Success Metrics

Company-Level RACI for Standards

The RACI below assigns accountability for each standard area. "R" = Responsible (does the work), "A" = Accountable (final decision), "C" = Consulted, "I" = Informed.

Standard AreaProduct OwnerDev Lead (kgray)Dev TeamABS (Azure Partner)External Contractors
Ratify & amend standardsAR, CCI
ADO/repo structure + branch policiesARCII
Dev environment standard (uv/ruff/pytest)IRRC
lat.md / CLAUDE.md authoringIRRC
Docs site contentCRRI
ADW pipeline operationsCRR
Azure RBAC + access governanceARIR
Azure cost monitoring (monthly)ARIC
Service principal + credential cleanup (quarterly)ARIR
GCP→Azure migration go/no-goAR, CCC
GitHub→ADO migration executionARRI
Contractor onboarding runbookARCI
Mars go-live (Aug 15)ARRC
Portal Recon Phase 4 deployARR

Standing governance cadence:

CadenceActivityOwner
DailyMorning digest review (ops digest daily)Dev Lead
WeeklyDigest review with PO; flag blockersPO + Dev Lead
MonthlyRBAC + DB roles access review; cost review per subscriptionDev Lead (PO accountable)
QuarterlyService principal + credential cleanup; standards retrospectiveDev Lead (PO accountable)
Half-yearlyGCP decision revisit (if no-go was recorded); tech-stack reviewPO + Dev Lead

Success Metrics

Metrics are only useful if they are measured at a fixed cadence and tied to a named owner. Record baselines at Day 90 and measure again at Day 180.

MetricBaseline (Day 90 target)Goal (Day 180)Measurement Method
Time to onboard a new developer (from first commit access to "green uv run pytest")< 1 business day< 4 hoursTimed during first real onboard; logged in ADO wiki
Time to scaffold a new repo (ADO repo + branch policies + CLAUDE.md + lat.md + first passing lat check)< 2 hours< 1 hourTimed against the Section A checklist
% owned repos on standard (uv + ruff + pytest + lat.md + CLAUDE.md + ADO branch policy)60%100%ops doctor or manual audit; tracked in ADO dashboard
Docs freshness (% of owned-project docs pages with a last-updated date within 30 days of most recent code change)70%90%docs-triggers.yaml coverage; manual spot-check monthly
Drift alerts actioned within 1 business day100%ops azure drift output; reviewed in morning digest; tracked in ADO work items
lat check pass rate across owned repos100%100%CI gate; any failure blocks merge
ADW pipeline utilization (% of ADO work items dispatched via ADW rather than direct manual commits)20%60%ops digest daily ADW section; work item adw tag count
Time from ADO work item creation to first deployed commit (ADW-dispatched items)< 4 hours< 2 hoursADO work item timestamps; digest ADW timing log
GCP monthly billCurrent (measure now)$0 (if go) or documented + frozen (if no-go)GCP billing export
Azure cost variance month-over-month< 20% spike triggers alert< 20% spike triggers alertops azure drift cost spike detection; digest AZURE GOVERNANCE block
Open security findings (RBAC/cred expiry)0 overdue (> 30 days past expiry)0 overdueops azure drift expiry findings; monthly access review
Contractor off-boarding completeness (all access revoked within 1 business day of engagement end)100%ADO access audit; DB role audit via ops db users list

G6. What Success Looks Like at 6 Months

At six months, the platform owner should be able to answer "yes" to all of the following without looking anything up:

  • A new developer is productive (commits merged, tests passing) within half a business day.
  • Every owned repo passes lat check in CI — no broken docs, no orphaned code refs.
  • The morning digest runs without manual intervention and the AZURE GOVERNANCE block shows real cost data.
  • GitHub is a read-only archive or fully decommissioned; all active repos live in ADO.
  • GCP is either decommissioned (if go) or has a dated, signed no-go ADR with an annual review on the calendar.
  • The Portal Reconciliation, Mars/Orbit, and Prefect ETL platforms each have a named owner in the RACI who reviews their section of the digest weekly.
  • No service principal or DB credential is within 30 days of expiry without a tracked remediation work item.

These are not aspirational — they are the minimum viable governance posture for a team adding headcount into an existing platform. The standards in sections A–F create the conditions; this section sequences them into an achievable path.