INTERACTIVE

Anatomy of Data Drift

A model learns one window of the world and then gets deployed into a world that keeps moving. This is six months of that, played day by day: the trials arriving, the check comparing them to the training data, and the outcomes landing two weeks late. Press play, pick a scenario, and change the test, the threshold, and the window to see what each one catches and what it misses.

SCENARIO

Day 1 · week 1

reference (training data) current window

Six months of checks

feature flagged as drifted checked, not drifted predicted conversion rate actual conversion rate, two weeks late what really happened