historical MAPE
Recorded competition result, not promoted as rebuild evidence.
← All work
ForecastingArchived project
DSC MCF ITB 2025
Claims forecasting where leakage control, lag correctness, and chronological evaluation matter more than adding another model.
Forecasting2026-08-29
DSC MCF ITB 2025
4.08%historical MAPE
3material pipeline defects
3model families
Quick read
Project summary
Problem
Time-series accuracy is meaningless if a pipeline can see the future or if lag logic silently changes which periods are evaluated.
My role
I reviewed the historical forecasting workflow and identified target-equivalent leakage, lag-propagation defects, and row loss that must be removed before a rebuild result can be compared fairly.
Result
Historical MAPE 4.08%, retained as a historical result until a leakage-safe rebuild proves comparable validity.
Stack
LightGBM · Prophet · Holt-Winters ETS · time-series validation
Status
Archived project
material pipeline defects
Leakage, lag propagation, and row-loss issues identified in review.
model families
LightGBM, Prophet, and Holt-Winters ETS.
Overview
This case study is framed as a forecasting validation lesson, not as a leaderboard trophy. The key evidence is the discovery of defects that can invalidate otherwise attractive accuracy numbers.
Problem
Time-series accuracy is meaningless if a pipeline can see the future or if lag logic silently changes which periods are evaluated.
Why it matters
A strong forecasting score is not credible if features can see the future or lag logic changes which periods are evaluated. Chronology has to be proven before accuracy is promoted.
My role
I reviewed the historical forecasting workflow and identified target-equivalent leakage, lag-propagation defects, and row loss that must be removed before a rebuild result can be compared fairly.
Product
The original competition workflow combined LightGBM, Prophet, and Holt-Winters ETS for claims forecasting. The portfolio focuses on the validation lessons from reviewing that workflow.
Data
Competition data is treated as restricted and remains outside the public portfolio until redistribution rights are confirmed.
Method
The review requires every feature and lag to be available before forecast time, and ensemble weights must not be optimized on the final holdout.
- Block features that recreate the target directly or indirectly.
- Build every lag only from observations available before forecast time.
- Prevent lag handling from dropping periods that do not require an unavailable lag.
- Keep final holdout evaluation separate from ensemble-weight tuning.
Results
Historical MAPE of 4.08% is retained as historical evidence. It is not relabeled as leakage-safe rebuild performance.
- Three material defects were identified: target-equivalent leakage, incorrect lag propagation, and row loss caused by lag handling.
- The historical workflow used LightGBM, Prophet, and Holt-Winters ETS.
Validation
A publishable rebuild must prove chronological feature availability before any accuracy comparison is accepted.
Architecture
The next architecture separates feature-availability rules, lag construction, training, ensemble tuning, and final holdout evaluation so temporal leakage can be tested directly.
- Feature availability rulesOnly information known before forecast time
- Lag constructionChronology-safe feature generation
- Model trainingLightGBM, Prophet, and Holt-Winters paths
- Ensemble tuningSeparate from the final holdout
- Chronological holdoutFinal leakage-safe evaluation gate
Tradeoffs
Keeping restricted competition data private reduces public reproducibility today, but it is preferable to publishing data without confirmed redistribution rights.
Limitations
The 4.08% MAPE cannot be treated as comparable rebuild evidence until the historical defects are removed and chronology is retested.
Live Product
There is no live product link in this case study. The value of the case is the validation review and the plan for a legally publishable rebuild.
Source
Restricted competition data and unverified source remain outside the public portfolio. This case study publishes only the reviewed claims that can be stated without exposing restricted material.
Technical status: Recorded competition project
Repository status: Restricted competition data and unverified source remain outside the public portfolio.
Next milestone
Reconstruct a legally publishable, leakage-safe forecasting path with feature-availability documentation, safe fixtures, model adapters, regression tests for the three historical defects, and chronological evaluation.