Prefect — Azure Resources & Access
Reference for all Azure resources, RBAC assignments, and Prefect secret blocks that back the Prefect ETL platform. Use this page to answer "where does X live?" and "who can touch Y?" questions.
Azure Subscriptions
Two Azure subscriptions are used:
| Subscription | ID | Purpose |
|---|---|---|
| Wasteology Development | 669a4356-6871-4f9b-9256-ba98d5a6eb4c | CieTrade ACI flow execution, source ACRs |
| Wasteology Production | a5a6818e-a0c0-4c6a-8b5c-21e91d75caa8 | Always-on apps, non-CieTrade ACI flow execution |
Azure Container Registries (ACRs)
| Registry | Subscription | Resource Group | Active Images | Status |
|---|---|---|---|---|
cietradeacr.azurecr.io | Development | prefect-resources | prefect:latest, wdp-imports:latest | Active — CieTrade flows only |
wasteologypipelinesacr.azurecr.io | Production | prefect-resources | enevo-etl:latest, goodwill-etl:latest, qbo-etl:latest, ach-remittance:latest | Active — new images go here |
enevoacr.azurecr.io | Development | prefect-resources | enevo-etl:latest | Superseded — mirrored to Prod ACR |
goodwillacr.azurecr.io | Development | prefect-resources | goodwill-etl:latest | Superseded — mirrored to Prod ACR |
wdppalantiriacr.azurecr.io | Production | wdp-palantiri-rg | wdp-palantiri, adw-agent, wg-orchestrator | Active — platform apps only |
Push new ETL images to wasteologypipelinesacr.azurecr.io.
Resource Groups
Development Subscription
| Resource Group | Contents | Purpose |
|---|---|---|
prefect-resources | cietradeacr, enevoacr, goodwillacr, ephemeral ACI instances | CieTrade flow execution |
wdp_import | wdpimport Container App, wdp-docs | WDP import pipeline |
enevo-etl-rg | enevo-etl-identity managed identity | Legacy Enevo identity |
Production Subscription
| Resource Group | Contents | Purpose |
|---|---|---|
prefect-resources | wasteologypipelinesacr, ephemeral ACI instances | Non-CieTrade flow execution (Enevo/Goodwill/QBO) |
wdp-palantiri-rg | wdppalantiriacr, wdp-palantiri, adw-agent, wg-orchestrator, tirion-agent VM, Key Vault, blob store | Always-on platform apps |
invoice-automation | SQL Server wasteology.database.windows.net | CieTrade source system (ABS-managed) |
Service Principal: prefect-workpool-sp
The service principal prefect-workpool-sp (AppId: a4ccc4ec-6368-49b8-bf79-89b6eadcd057) authenticates all ACI operations. Its credentials are stored in Prefect secret blocks and referenced by deployments.
| Scope | Role | Purpose |
|---|---|---|
Dev prefect-resources RG | Contributor | Spin CieTrade ACI containers, pull from Dev ACRs |
Prod prefect-resources RG | Contributor | Spin non-CieTrade ACI containers |
wasteologypipelinesacr | AcrPull | Pull enevo-etl, goodwill-etl, qbo-etl images |
RBAC: Developer Access
| Person | Scope | Role | What they can do |
|---|---|---|---|
| kgray | Production subscription | Contributor + Data Factory Contributor | Create/manage any resource; cannot assign roles |
| kgray | Prod prefect-resources RG | User Access Administrator | Grant any role within this RG without ABS |
| Shane (Sstclair) | Prod prefect-resources RG | Contributor | Manage ACI containers in this RG |
| Shane (Sstclair) | wasteologypipelinesacr | AcrPush | Push and pull flow images |
| Shane (Sstclair) | Prefect Cloud workspace | Editor | View/trigger/deploy flows; cannot manage workspace settings |
Note: Shane has no access to the Development subscription. All new flows for Shane go to Prod prefect-resources.
Prefect Secret Blocks
Secret blocks store credentials in Prefect Cloud — deployments reference them with {{ prefect.blocks.secret.<name> }} in job_variables. Modifying a shared block affects every deployment that references it.
| Block Name | Value / Description | Used By |
|---|---|---|
azure-subscription-id | Dev subscription ID (669a4356) | All CieTrade deployments — DO NOT CHANGE |
azure-resource-group | prefect-resources (Dev RG) | All CieTrade deployments — DO NOT CHANGE |
azure-sp-client-id | prefect-workpool-sp AppId | All deployments |
azure-sp-client-secret | prefect-workpool-sp secret | All deployments |
azure-sp-tenant-id | Wasteology Azure AD tenant | All deployments |
azure-prod-subscription-id | Prod subscription ID (a5a6818e) | Non-CieTrade flows only |
azure-prod-resource-group | prefect-resources (Prod RG) | Non-CieTrade flows only |
azure-prod-acr-login-server | wasteologypipelinesacr.azurecr.io | Non-CieTrade flows only |
Note: Non-CieTrade flows (enevo, goodwill, qbo) do NOT use azure-subscription-id — they hardcode the Prod subscription ID directly in deployment job_variables. This prevents a single secret block change from accidentally redirecting CieTrade flows.
ACI Quota
ACI CPU quota in Dev prefect-resources: 10.0 CPU cores (monitored every 4 hours by the aci-quota-monitor flow). If the cietrade-aci-pool concurrency limit (30) and actual CPU quota conflict, Prefect queues excess runs rather than erroring.
A quota increase to ≥ 20 cores on the Production subscription is recommended before migrating CieTrade flows. Contact kgray to initiate an ABS request.