Central Bank Indicators Pipeline

Architecture

Problem

Companies that import goods, price products, or hold indexed debt need daily updated SELIC, IPCA and FX data, but collecting and organizing this manually is slow and error-prone.

Solution

Automated ELT pipeline: extraction from the Central Bank's SGS API with Python, load into BigQuery's raw layer (upsert), and star-schema modeling (fact + dimensions) with dbt. Orchestrated with GitHub Actions, running every business day at 6am (Brasília time).

Results

• Idempotent load via incremental MERGE, with no data duplication
• 19 automated data quality tests with dbt
• Fully automated daily run via GitHub Actions