Analytics Pipeline: PostgreSQL + Python + Dashboard

We've been accumulating raw event data in MySQL for some time now, but it just sits there — there's no real analytics on top of it, and we can't answer basic questions about how the product is being used. We want to build a full analytics pipeline that takes this raw data, transforms it into clean modeled tables in PostgreSQL, and displays key metrics on a BI dashboard accessible to the whole team.

The core work is an ETL pipeline that extracts data from MySQL, cleans and aggregates it, and loads it into PostgreSQL, with well-designed fact and dimension tables for analyzing events, users, and revenue. On top of that we need scheduled jobs — daily and hourly aggregations via cron or Airflow — and a dashboard in Grafana or Metabase connected to PostgreSQL. The metrics that matter to us are DAU, MAU, retention, ARPU, and funnel conversion, and we also need automatic data quality checks (row counts and null validation) so we can trust the dashboard.

The stack is oriented toward Python, and everything should be spun up via Docker Compose for a clean local environment, so that after handoff we can run and extend the pipeline ourselves.

Components:
— 1. ETL — extraction from MySQL, transformation (cleaning, aggregation), loading into PostgreSQL
— 2. Data models — fact/dimension tables for analyzing events, users, revenue
— 3. Scheduler — daily/hourly aggregations (cron or Airflow)
— 4. Dashboard — Grafana or Metabase connected to PostgreSQL
— 5. KPI — DAU, MAU, retention, ARPU, funnel conversions
— 6. Data quality control — row counts and null checks

Stack:
— Python 3 (pandas, SQLAlchemy, psycopg2)
— PostgreSQL
— Grafana or Metabase
— Docker Compose for local deployment