WDP Palantiri
Wasteology Data Platform - Monitoring & Operations UI
Palantiri is a Streamlit-based web application for managing and monitoring the Wasteology data platform. It provides a unified interface across 11 specialized pages spanning monitoring, AI agent management, data management, and financial operations.
Access the live application at wdp-palantiri.politegrass-38e79dd2.eastus.azurecontainerapps.io
Architecture
Key Features
Migration Tool
Migrate data between PostgreSQL and Azure SQL with Parquet or Sling modes
Data Browser
Browse schemas, tables, and preview data across all connected databases
Prefect Monitoring
Monitor flow runs, view task breakdowns, and trigger migrations manually
dbt Lineage
Visualize dbt model DAG with interactive upstream/downstream highlighting
CI/CD Dashboard
Live operations view for Prefect ingestion health and dbt production pipeline status
Project Overseer
Unified portfolio monitoring with pipeline alerts, 30-day health trends, and digest tasks
Tirion AI Agent
Real-time monitoring dashboard for the Tirion autonomous AI agent and its OODA loop
DB Access Manager
Visual RBAC management for all Wasteology PostgreSQL databases with dry-run diffing
QuickBooks / ACH
ACH validation, contact management, and QBO financial views across all 11 entities
Enevo Sensors
IoT sensor data browser and ETL health monitor for active Enevo customer containers
Portfolio Intelligence
Strategic portfolio knowledge graph and natural language query interface
Supported Databases
Palantiri connects to four Wasteology database environments:
| Database | Type | Environment |
|---|---|---|
| PostgreSQL Production | PostgreSQL | ✓Production |
| PostgreSQL Development | PostgreSQL | ℹDevelopment |
| Azure SQL Production | Azure SQL | ✓Production |
| Azure SQL Development | Azure SQL | ℹDevelopment |
Quick Start
Running Locally
# Clone the repository
git clone https://dev.azure.com/wasteology/Wasteology/_git/wdp-palantiri
cd wdp-palantiri
# Install dependencies
uv sync
# Configure environment
cp .env.example .env
# Edit .env with your database credentials
# Run the application
uv run streamlit run app.py
The application will be available at http://localhost:8501.
Azure Deployment
Palantiri is deployed via Azure Container Apps and automatically deployed via ADO CI pipeline on push to main. For manual deployment:
cd ~/projects/orchestration/wdp-palantiri && ./scripts/build-and-push.sh --deploy
Authentication
Palantiri supports two authentication modes:
- Password Protection - Simple password via
STREAMLIT_PASSWORDenvironment variable - Azure AD - Enterprise authentication via MSAL (configured via
AZURE_AD_*environment variables)
For local development, leave STREAMLIT_PASSWORD unset to bypass authentication.
Next Steps
- Migration Tool - Learn how to migrate data between databases
- Data Browser - Explore your database schemas
- Prefect Flows - Monitor orchestration pipelines
- dbt Lineage - Visualize data model relationships
- CI/CD Dashboard - Live ingestion and pipeline health
- Project Overseer - Portfolio monitoring and digest tasks
- Tirion Agent - AI agent status and activity feed
- DB Access Manager - PostgreSQL RBAC management
- QuickBooks - ACH validation and financial views
- Enevo Sensors - IoT sensor data and ETL health
- Portfolio Intelligence - Knowledge graph and NL queries