warehouse core
Planned normalized BigQuery core.
← All work
Data engineeringLive dashboard
PanganLens Indonesia
Public-benefit food-price analytics designed around provenance, idempotent loading, conflict handling, revision history, and warehouse quality.
Data engineering2026-09-07
PanganLens Indonesia
3NFwarehouse core
18:00 WIBplanned refresh
Fail closedquality stance
Quick read
Project summary
Problem
A food-price dashboard is only trustworthy when every observation has an inspectable source, business identity, quality decision, and revision path.
My role
I am designing the acquisition, warehouse, data-quality, revision, and reporting architecture. Live acquisition remains the next verification gate, so no production coverage claim is made yet.
Result
The public production route is live, while live acquisition, warehouse coverage, and current data behavior remain separate verification gates.
Stack
PIHPS Bank Indonesia · BigQuery · 3NF data modeling · data quality controls
Status
Live dashboard
planned refresh
Daily schedule after acquisition verification.
quality stance
Conflicts and invalid records do not silently enter reporting marts.
Overview
PanganLens is a public-benefit food-price project where traceability and data quality come before dashboard polish.
Problem
A food-price dashboard is only trustworthy when every observation has an inspectable source, business identity, quality decision, and revision path.
Why it matters
Public food-price reporting is only trustworthy when observations keep their source, business identity, quality decision, and revision history from acquisition through the reporting layer.
My role
I am designing the acquisition, warehouse, data-quality, revision, and reporting architecture. Live acquisition remains the next verification gate, so no production coverage claim is made yet.
Product
The public dashboard route is deployed and reachable. The portfolio keeps deployment availability separate from claims about live PIHPS acquisition, warehouse coverage, and current data quality.
Data
PIHPS Bank Indonesia is the preferred primary-source direction for the MVP.
- Machine-readable official access is preferred before report downloads.
- HTML extraction is a last resort rather than the default ingestion path.
- Each accepted observation should retain source provenance and revision history.
Method
Deterministic business keys and record hashes are designed to distinguish inserts, repeated records, conflicts, and revisions.
- Separate staging, normalized dimensions and facts, revisions, quarantine, and load-audit layers in BigQuery.
- Idempotent merge behavior before a curated reporting mart is exposed.
Results
The current evidence combines a verified public production route with architectural and process evidence. No dashboard impact metric, live PIHPS acquisition, or warehouse coverage is claimed yet.
- Public production route returned HTTP 200 on 7 September 2026.
- Preferred source direction is PIHPS Bank Indonesia.
- Normalized 3NF BigQuery core is the documented warehouse direction.
- Daily 18:00 WIB refresh remains a plan that begins only after acquisition verification.
Validation
Acquisition, schema, idempotency, conflict handling, and revision behavior must pass before downstream dashboard claims are promoted.
Architecture
The planned data path moves from source acquisition to staging, normalized core, revisions and quarantine, then to a curated reporting mart.
- PIHPS acquisitionPreferred official source direction
- Staging and validationSchema, identity, and quality checks
- Normalized 3NF coreDimensions, facts, and business keys
- Revisions and quarantineConflicts remain outside curated output
- Reporting martCurated layer before dashboard claims
Tradeoffs
The project deliberately delays visual polish and broad source coverage until one official acquisition path is reproducible and reviewable.
Limitations
The public dashboard deployment is verified, but live PIHPS acquisition, warehouse coverage, and current dashboard data behavior have not been independently verified in this portfolio run.
Live Product
The public PanganLens production route returned HTTP 200 on 7 September 2026 and is published as a live product link. This verifies deployment availability only; it does not promote unverified live acquisition or warehouse coverage claims.
Source
A GitLab project is reachable, but its verified remote state currently contains only a minimal README. It is therefore not used as implementation evidence for this case study.
Technical status: Production dashboard live; data acquisition verification pending
Repository status: A GitLab project exists but was not used as implementation evidence because the verified remote state contains only a minimal README.
Next milestone
Establish one live-reviewed PIHPS acquisition contract, guarded provider adapter, BigQuery DDL, idempotent merge proof, revision handling, and only then a curated reporting mart.