Early warning for live games on Steam
Review 10% of your catalogue. Catch most of next year's collapses.
A risk score for every live Steam game: will it lose half its players, relative to the market, within a year? Each score comes with its reasons.
| Rank | Game | Players then | Risk | Tier | Kept vs market |
|---|
The problem
Big drops are easy to miss until it is too late
Each quarter, publishers decide which live games get updates, sales and marketing. A game about to lose half its audience needs that decision early.
Who it is for
Live-ops and portfolio leads, marketing analysts, storefronts and investors.
Why simple rules fail
- Player floors miss a drop from 5,000 to 800.
- Trends confuse post-sale spikes with decline.
- Year-on-year ignores the market: −26% in 2018, +34% in 2020.
How it works
From player counts to an explained watch list
Collect
303.6M Steam player counts for 2,000 apps (2017–2020), store data, monthly Twitch stats.
Clean
1.94M game-days, real games only, Twitch links checked by hand.
Learn
57 signals from each 90-day window, at 21 monthly start dates.
Score
A calibrated probability, a tier and three reasons per game.
What counts as a collapse
Players in months 6–12 fall below half of where the typical live game ends up. Only live games (10+ players) are scored.
How it was tested
Retrained each month of 2019 on outcomes known by then, then scored games it had never seen.
Results
A short review list that holds most collapses
65 of 1,559 games collapsed after February 2019. Drag the slider: how many would a review of the riskiest games have caught?
February 2019 backtest; source: reports/decline_metrics.json.
It holds up every month
ROC-AUC per test month, 2019.
The probabilities mean what they say
Actual vs expected collapses per tier, Feb 2019.
| Tier | Games | Collapsed | Predicted | Of all collapses |
|---|---|---|---|---|
| Watch listtop 10% | 155 | 27% | 22% | 65% |
| Monitornext 20% | 312 | 3.2% | 4.0% | 15% |
| Stableother 70% | 1,092 | 1.2% | 2.4% | 20% |
Where it is strong, and where the evidence is thin
ROC-AUC (0.5 = coin flip) with 95% intervals, Feb 2019.
Only 6 big games collapsed, too few to judge. On launches it barely beats the trend.
Reasons for every flag
Spiky bursts, weak floors and falling trends drive the risk
Each score splits exactly into per-signal contributions. The top three become the game's reasons.
Tom Clancy's The Division · 4,098 players
Watch list, February 2019 · kept 16% a year later
Airstrike HD · 33 players
Watch list, February 2019 · kept 1% a year later
Who collapses?
Collapse rate from a Dec 2017 start (all games: 11%).
Twitch adds nothing
Adding Twitch changes ROC-AUC by −0.006, which is noise. Steam data alone is enough.
Age and genre matter, price model does not
Free-to-play and paid: 11% each. Casual 25%, Indie 17%. Games over 5 years: 1%.
Explore the watch list
Every scored game, with its risk and reasons
Forecast: scored on May–Aug 2020, looking into 2021, not yet verifiable. Backtest: Feb 2019, with real outcomes.
Limits
What the product does not do
Big games
Only 6 of 170 collapsed; the interval (0.46–0.99) is too wide to claim anything.
New launches
The trend rule ranks them almost as well (0.75 vs 0.77).
Causes
Reasons show which signals raised a score, not why a game declined.
Scope
2,000 mostly popular Steam apps, 2017–2020. Safe games are slightly over-predicted.
Technical report
Method in brief
Full details are in the repository (reports/technical_report.md).
- Steam data
- Steam Games Dataset, Wannigamage et al., Mendeley Data, 2020 (CC BY 4.0). 2,000 apps, Dec 2017 – Aug 2020, aggregated to daily means.
- Twitch data
- David Boyd – SullyGnome.com, monthly game statistics 2017–2020, used with permission.
- Cleaning
- 1.94M game-days, 1,851 games; 6 wrong Twitch links removed.
- Target
- Players in days 180–365 vs days 0–89, relative to the median live game. Below 0.5 = collapse.
- Training
- 21 monthly start dates; each test uses only outcomes known before it.
- Model
- Calibrated random forest (500 trees), compared with logistic regression, boosting and simple rules.
- Explanations
- Exact tree-path decomposition, grouped into 14 themes.
Learning outcomes
What we learned building it
The target defines the product
"Stays above 10 players" scored 0.98 but only flagged near-empty games.
Always compare with a baseline
Every result is set against rules a publisher could apply alone.
Test the way it will be used
A monthly backtest showed that more history helps and the market swings.
Check every join
A fake "primary genre" and wrong Twitch links changed results until caught.