Invoice Projects — GCP Infrastructure

Scope: GCP infrastructure only — Cloud Run, Cloud Functions, Pub/Sub, VPC, Artifact Registry, Document AI, Cloud Storage. Does not cover application code, frontend, or database internals.

GCP Project

Both projects share a single GCP project:

Field	Value
Production Project	`academic-torch-405913`
Dev Project	`trashy-dev-481621`
Region	`us-central1`
IaC	Terraform (prod + dev environments)
Build Service	Google Cloud Build (via `deploy.sh`)
Secret Management	GCP Secret Manager

Invoice Listener

Automated invoice ingestion pipeline. Watches email and OneDrive for incoming invoices, processes them through Document AI, and feeds results to invoice-management.

Cloud Run Services

Always-on HTTP endpoints that receive Pub/Sub push messages:

Service Name	Purpose
`wasteololgy-email-handlers`	Processes emails pushed from Pub/Sub
`wasteololgy-onedrive-handlers`	Processes OneDrive files pushed from Pub/Sub

note

Service names contain a typo — wasteololgy (double "l", missing "g") instead of wasteology. This is the actual resource name in GCP.

Cloud Run Jobs

Scheduled tasks that run on a timer, do their work, and exit:

Job Name	Schedule	What It Does
`wasteologyinvoice-email-queued-listener`	Every 10 min	Polls email inbox for new invoices
`wasteologyinvoice-onedrive-listener-queued`	Every 1 hour	Polls OneDrive for new invoice files
`wasteologyinvoice-error-email-processor`	Every 1 hour	Retries previously failed/errored emails

VPC & Networking

All Cloud Run resources are attached to a dedicated VPC:

Resource	Name
VPC	`wasteology-invoice-processing-vpc`
Subnet	`listeners`
Egress mode	`all-traffic`

The YAML configuration on every Cloud Run resource:

run.googleapis.com/vpc-access-egress: all-traffic
run.googleapis.com/network-interfaces:
  - network: wasteology-invoice-processing-vpc
    subnetwork: listeners

All outbound traffic routes through the VPC, which requires Cloud NAT for internet access. The VPC has no private resources inside it — it exists purely to provide a stable outbound IP for external API calls (Microsoft Graph, Azure SQL, Document AI, etc.).

Optimization

Since there are no private resources in the VPC, this could be simplified by either removing the VPC entirely (use Cloud Run default egress) or switching to private-ranges-only egress to avoid Cloud NAT costs.

Artifact Registry

us-central1-docker.pkg.dev/academic-torch-405913/wasteology-invoice-processing/

Container images are built locally and pushed here. Three image types are built from the same repo (HttpApp, ListenerApp, QueueApp).

Document AI

Field	Value
Processor ID	`9e41876640558aee`
Location	`us` (via `us-documentai.googleapis.com`)
Purpose	OCR + entity extraction from invoice PDFs

Cloud Storage

Bucket	Purpose
`invoice_inference_json_output`	Document AI JSON output + processed PDFs. Handoff point — listener writes here, invoice-management reads from here.

BigQuery

Dataset.Table	Purpose
`ParsedInvoices.responses`	Raw Document AI entity extraction results

Invoice Management (Trashy)

Full-stack invoice management application. Receives invoices from the listener pipeline and provides a web UI for managing them.

Cloud Run Services

Service Name	Purpose
`invoice-management-backend`	FastAPI application (~66 REST endpoints)
`invoice-management-frontend`	Nginx serving React build (port 8080)

Cloud Functions

Three Cloud Functions form an event-driven processing pipeline, each triggered by a Pub/Sub topic:

Step	Cloud Function	Trigger Topic	What It Does	Publishes To
1	`doc_ai_processor`	`invoice-uploaded`	Calls Document AI for OCR	`invoice-extracted`
2	`db_inserter`	`invoice-extracted`	Inserts extracted data into Cloud SQL	`invoice-persisted`
3	`webservice_caller`	`invoice-persisted`	Calls external APIs (address lookup, assignment)	—

The chain starts when the listener either:

POSTs to the backend's /invoices/ingest endpoint, or
Publishes to the send-to-trashy Pub/Sub topic

Pub/Sub Topics

Topic	Publisher	Subscriber
`send-to-trashy`	invoice-listener	invoice-management
`invoice-uploaded`	Backend ingest endpoint	`doc_ai_processor` function
`invoice-extracted`	`doc_ai_processor`	`db_inserter` function
`invoice-persisted`	`db_inserter`	`webservice_caller` function

Cloud SQL

Field	Value
Type	PostgreSQL
Project	`academic-torch-405913`
Schema	`trashy`

Cloud Storage

Bucket	Purpose
`wasteology-invoices-pdf-dev` / `wasteology-invoices-pdf-prod`	Primary PDF storage
Legacy GCS bucket	Fallback read location for older files

Storage strategy: GCS first, fallback to Azure Blob for historical PDFs.

How the Two Projects Connect

  invoice-listener                              invoice-management
 ┌──────────────────────┐                    ┌──────────────────────────┐
 │                      │                    │                          │
 │  Cloud Run Jobs      │                    │  Cloud Run Services      │
 │  (scheduled polling) │                    │  (backend + frontend)    │
 │       │              │                    │       ▲                  │
 │       ▼              │                    │       │                  │
 │  Cloud Run Services  │                    │  Cloud Functions         │
 │  (Pub/Sub handlers)  │                    │  (processing chain)      │
 │       │              │                    │       ▲                  │
 │       ▼              │    3 handoffs      │       │                  │
 │  Document AI         │ ──────────────────>│                          │
 │       │              │                    │  1. Pub/Sub topic        │
 │       ▼              │                    │     "send-to-trashy"     │
 │  GCS bucket ─────────│── shared bucket ──>│                          │
 │  (invoice_inference_ │                    │  2. GCS bucket           │
 │   json_output)       │                    │     (invoice_inference_  │
 │       │              │                    │      json_output)        │
 │       ▼              │  HTTP POST         │                          │
 │  BigQuery            │ ──────────────────>│  3. POST /invoices/ingest│
 │                      │                    │                          │
 └──────────────────────┘                    └──────────────────────────┘

The Three Handoff Points

#	Mechanism	From	To
1	Pub/Sub topic `send-to-trashy`	Listener publishes after processing	Management subscribes, triggers Cloud Function chain
2	GCS bucket `invoice_inference_json_output`	Listener writes Document AI JSON + PDFs	Management reads for display and storage
3	HTTP POST to `/invoices/ingest`	Listener calls backend API directly	Management ingests and starts processing chain

Complete Resource Inventory

Shared Resources

Resource Type	Name / ID	Used By
GCP Project	`academic-torch-405913`	Both
Document AI Processor	`9e41876640558aee`	Both (listener directly, management via Cloud Functions)
GCS Bucket	`invoice_inference_json_output`	Listener writes, Management reads
Pub/Sub Topics	`send-to-trashy`, `invoice-uploaded`, `invoice-extracted`, `invoice-persisted`	Pipeline chain

Invoice Listener Resources

Resource Type	Name	Notes
Cloud Run Service	`wasteololgy-email-handlers`	Typo in name
Cloud Run Service	`wasteololgy-onedrive-handlers`	Typo in name
Cloud Run Job	`wasteologyinvoice-email-queued-listener`	10 min schedule
Cloud Run Job	`wasteologyinvoice-onedrive-listener-queued`	1 hour schedule
Cloud Run Job	`wasteologyinvoice-error-email-processor`	1 hour schedule
VPC	`wasteology-invoice-processing-vpc`	All-traffic egress
Subnet	`listeners`	Inside VPC
Cloud NAT	(associated with VPC)	Required for outbound internet
Artifact Registry	`wasteology-invoice-processing/`	Container images
BigQuery	`ParsedInvoices.responses`	Document AI results

Invoice Management Resources

Resource Type	Name	Notes
Cloud Run Service	`invoice-management-backend`	FastAPI
Cloud Run Service	`invoice-management-frontend`	Nginx + React
Cloud Function	`doc_ai_processor`	Pub/Sub triggered
Cloud Function	`db_inserter`	Pub/Sub triggered
Cloud Function	`webservice_caller`	Pub/Sub triggered
Cloud SQL	PostgreSQL instance	`trashy` schema
GCS Bucket	`wasteology-invoices-pdf-dev` / `prod`	Primary PDF storage

Terraform (Infrastructure as Code)

Both projects are managed by Terraform with separate environment configurations:

Environments

Environment	GCP Project	Terraform State
Production	`academic-torch-405913`	GCS `wasteology-terraform/terraform/state/prod`
Dev	`trashy-dev-481621`	GCS `wasteology-terraform/terraform/state/dev-v2`

What Terraform Manages

Resource	Prod	Dev
Cloud SQL (PostgreSQL 17)	Regional HA, deletion protection ON	Zonal, deletion protection OFF
Cloud Run Services	Backend, frontend, 4 handler services	Same, with corrected names (no typo)
GCS Buckets	`invoice_inference_json_output`, `wasteology-invoices-pdf`, `wasteology-invoices-2`	Prefixed dev equivalents
Document AI	Processor `9e41876640558aee`	Dev processor
Pub/Sub Topics	5 topics	Same
VPC + Private Network	VPC peering for Cloud SQL	Same

info

Terraform manages infrastructure shape but not container images. Image updates happen via the deploy script. A lifecycle { ignore_changes = [image] } block prevents Terraform from reverting deployments.

Secret Manager

Secrets are stored in GCP Secret Manager:

Backend Secrets (8)

Secret Name	Purpose
`invoice-management-database-url`	PostgreSQL connection string
`invoice-management-entra-client-secret`	Azure AD auth
`invoice-management-azure-storage-cs`	Azure Blob Storage (prod only)
`invoice-management-msgraph-client-secret`	Microsoft Graph API
`invoice-management-address-api-password`	Address API credentials
`invoice-management-address-api-username`	Address API credentials
`invoice-management-db-user`	PostgreSQL user
`invoice-management-db-pass`	PostgreSQL password

Handler Secrets (7)

Secret Name	Purpose
`handler-address-api-password`	Address API
`handler-address-api-username`	Address API
`handler-msgraph-client-secret`	Microsoft Graph
`handler-queue-connection-string`	Azure Storage Queue
`handler-storage-connection-string`	Azure Blob Storage
`handler-trashy-api-client-secret`	Trashy API service principal
`handler-wasteology-connection-string`	Azure SQL ODBC

Deployment

Deployments use Google Cloud Build as the build service, initiated manually via a deploy script (gcp_deployment/deploy.sh).

note

Cloud Build YAML templates (cloudbuild.backend.yaml, cloudbuild.frontend.yaml) exist in the repo for future automated trigger-based CI/CD. Currently there are no git-push triggers configured.

Invoice Listener

# Manual: build locally, push, replace YAML
docker build -t us-central1-docker.pkg.dev/academic-torch-405913/wasteology-invoice-processing/<image>:<tag> .
docker push us-central1-docker.pkg.dev/academic-torch-405913/wasteology-invoice-processing/<image>:<tag>
gcloud run services replace cloudrun/<service>.yaml
gcloud run jobs replace cloudrun/<job>.yaml

Invoice Management

# deploy.sh handles the full lifecycle
./gcp_deployment/deploy.sh --all              # Deploy everything
./gcp_deployment/deploy.sh --backend          # Backend only
./gcp_deployment/deploy.sh --frontend         # Frontend only
./gcp_deployment/deploy.sh --function <name>  # Single Cloud Function

What deploy.sh does:

Enables required GCP APIs
Sets up VPC + Cloud NAT with static outbound IP
Creates/updates secrets in Secret Manager
Grants IAM roles to the compute service account
Submits Docker builds to Cloud Build (gcloud builds submit)
Deploys Cloud Run services, jobs, and Cloud Functions

Cloud Build Pipeline

Source Upload → Cloud Build → Docker Build → Artifact Registry → Cloud Run Deploy

Known Infrastructure Issues

#	Issue	Project	Impact
1	Secrets hardcoded in Cloud Run YAMLs	Listener	Management uses Secret Manager; listener still uses plaintext env vars
2	Typo in Cloud Run service names (`wasteololgy`)	Listener	Cosmetic; dev environment has corrected names
3	No automated CI/CD triggers	Both	Builds use Cloud Build but are initiated manually
4	VPC with `all-traffic` egress + Cloud NAT	Listener	Potentially unnecessary cost
5	No Alembic/migration tooling	Management	Schema changes to Cloud SQL are manual

External Dependencies

Services the GCP infrastructure connects to outside of GCP:

Service	Endpoint	Used By
Azure SQL Server	`wasteology.database.windows.net`	Listener
Azure Blob Storage	`pdfinvoices.blob.core.windows.net`	Listener (legacy)
Microsoft Graph API	`graph.microsoft.com`	Listener (email, OneDrive, SharePoint)
Address API	`api.politeground-1242c12a.centralus.azurecontainerapps.io`	Both
Microsoft Entra ID	Azure AD tenant	Management (auth)

GCP Project​

Invoice Listener​

Cloud Run Services​

Cloud Run Jobs​

VPC & Networking​

Artifact Registry​

Document AI​

Cloud Storage​

BigQuery​

Invoice Management (Trashy)​

Cloud Run Services​

Cloud Functions​

Pub/Sub Topics​

Cloud SQL​

Cloud Storage​

How the Two Projects Connect​

The Three Handoff Points​

Complete Resource Inventory​

Shared Resources​

Invoice Listener Resources​

Invoice Management Resources​

Terraform (Infrastructure as Code)​

Environments​

What Terraform Manages​

Secret Manager​

Backend Secrets (8)​

Handler Secrets (7)​

Deployment​

Invoice Listener​

Invoice Management​

Cloud Build Pipeline​

Known Infrastructure Issues​

External Dependencies​

GCP Project

Invoice Listener

Cloud Run Services

Cloud Run Jobs

VPC & Networking

Artifact Registry

Document AI

Cloud Storage

BigQuery

Invoice Management (Trashy)

Cloud Run Services

Cloud Functions

Pub/Sub Topics

Cloud SQL

Cloud Storage

How the Two Projects Connect

The Three Handoff Points

Complete Resource Inventory

Shared Resources

Invoice Listener Resources

Invoice Management Resources

Terraform (Infrastructure as Code)

Environments

What Terraform Manages

Secret Manager

Backend Secrets (8)

Handler Secrets (7)

Deployment

Invoice Listener

Invoice Management

Cloud Build Pipeline

Known Infrastructure Issues

External Dependencies