The Data Scientist: Who Is At Risk?

It is a Thursday in the fifteenth week, and Priya has the file everyone has been waiting for open on her screen: Aisha’s nightly feature table, one row per customer, 400 rows, refreshed at 2am without anyone touching it. Beside it sits Maya’s labelled sample — accounts a human went through by hand and marked as genuinely leaving, genuinely fine, or at risk for a reason nobody could act on. Underneath both is Ben’s finding that leavers start fading about ten weeks before they cancel, which is the only reason there is anything to predict at all.

This is the module most people think the whole course is about, so it is worth being blunt straight away. The model Priya builds this fortnight is one link in a chain of twelve, and most of what determines whether it is any good was settled before she wrote a line of code: by Sofia, choosing four indicators and refusing the rest; by Maya, writing down what “at risk” actually means; by Ben, finding the warning window and noticing that Corvex Manufacturing appeared twice under two spellings. Priya’s craft is real and it matters. It is also downstream of three people’s judgement, and it cannot rise above it.

What lands on Priya’s desk

The feature table is Aisha’s, from module 7: one row per customer, refreshed nightly. The labels are Maya’s, from module 5: the answer key, and the ceiling on how correct anything can be. The ten-week warning window is Ben’s, from module 6, and it fixes what Priya is predicting — not “will this account ever leave” but “is this account entering the fade that historically precedes a cancellation” — along with the segment cuts he found useful, plan and account size among them. And the four indicators are Sofia’s, from module 4: login frequency, core-feature usage, unused seats, failed payments. A behaviour nobody chose to collect in month one is not a difficult feature now; it is an absence.

So the question Priya owns is narrower than people imagine. Not why are customers leaving, which Ben answered, and not what do we do about it, which Jade owns in module 11. Just: given what we know this morning, how likely is each account to be on its way out, and who should a human ring first.

What a Data Scientist actually does

The popular picture of this job is somebody inventing a clever algorithm. The real job is closer to being a careful referee — the algorithms are off-the-shelf and free, and anyone can fit one in an afternoon. What separates a model that works in production from one that embarrasses everybody is unglamorous: making sure it learns from information that will exist at the moment you need a prediction, testing it on data it has never seen, and choosing the error trade-off on business grounds rather than mathematical ones.

And one honesty, repeated to every audience: the model does not know who will churn. It has looked at accounts that left and accounts that stayed, found the patterns that distinguished them, and applies those patterns to today’s accounts to produce an estimate of risk. When it is right, it is because this year’s leavers resemble last year’s. When the world changes — as it did at Northwind in March — it will be confidently wrong for a while, and nothing about its output will look any different.

The software on Priya’s desk

A Jupyter notebook keeps code, output and reasoning on the same page, which is why experiments live there: a colleague can re-run the thing and disagree with it rather than take it on trust. scikit-learn is the standard Python library for models of this kind, and it makes the mechanical part — train a simple baseline, train a random forest, score both on data the model has never seen — genuinely straightforward. That ease is exactly why the judgement discussed in this module matters so much more than the code.

MLflow is the logbook. It records every training run with its settings and its scores, so that “the model” is a specific recorded version rather than whatever happened to be open on a laptop in March — which also means a result can be reproduced when somebody eventually asks. The last screen is the important one: the precision-and-recall trade-off drawn as a curve, shared with Maya. It is presented in a business tool, to a business audience, because where to sit on that curve was never a technical decision.

The two decisions that decide everything

One: what the model is allowed to learn from

Everything on the candidate list is technically available, and that is precisely the trap. Available is not the same as usable, and two candidates would poison the model in completely different ways.

Start with the honest ones. A feature is just a column: the trend in logins over the last eight weeks, the trend in core-feature usage, the proportion of licensed seats anyone has actually signed into, the count of failed payments last quarter. Each is a number that exists for every customer, every morning, whether or not anything is wrong. Ben’s ten-week window shapes how they are built — a trend measured over eight weeks catches a slide that began ten weeks before a cancellation; a snapshot of last Tuesday catches nothing.

Now the poisoned one. Somewhere in the history sits a field recording that a customer opened a support ticket containing the word “cancel”. It is available, it is in the warehouse, and it predicts churn almost perfectly. Put it in the model and the test score is spectacular: an AUC of 0.97, the sort of number that gets screenshotted into a steering committee deck.

It is worse than worthless. By the time somebody types “cancel” into a ticket, they are already leaving. The feature is not predicting the future; it is reporting the present in the past tense. A model built on it sits silent through the whole ten-week fade — the only period in which anyone could have done something — then lights up in the week the decision has already been made. It predicts the past accurately and buys nobody a single day of warning.

Priya finds it the way you always find it — by validating on genuinely future data. She trains on accounts up to a cut-off date and scores the weeks after it, simulating what production will feel like: no knowledge of what happened next. The 0.97 collapses to 0.61, barely better than guessing. The feature comes out, and the honest model settles at an AUC of 0.84 — a genuinely useful number that will never impress anybody in a slide.

The second poisoned candidate fails a different test. Somebody has helpfully joined in a bought-in dataset carrying demographic attributes of each customer company’s owners. It is technically available. It is irrelevant to whether a business renews workflow software. And using it would mean scoring customers on the characteristics of the people who own them — discriminatory whether or not it improves accuracy, and something Elena would strike out in module 9 anyway. Priya strikes it first, because “governance will catch it” is not a control, it is a hope.

Both exclusions come from two questions worth stealing whatever your job is. First: would this value exist, in this form, at the moment I need the prediction? If not, it is leakage, however well it scores. Second: could I explain its use to the customer it describes? If not, it is out, however well it scores.

Two disciplines sit underneath all this. The model is trained on one group of accounts and tested on a different group it has never seen, because a model scored on what it memorised always looks excellent and tells you nothing. And Priya fits two models, not one: a logistic regression baseline — simple, explainable, a weighted sum you can read on a page — and a random forest, hundreds of small decision rules each voting on the answer. The forest wins by about six points of AUC, and the comparison is the point. Without a baseline, “the model achieves 0.84” floats in space; with one, it is a claim about what the complexity bought. Sometimes the answer is nothing, and then you ship the simple thing.

Two: where to draw the line

This is the most important idea in the module, and the least technical. Every account gets a risk score each night. A threshold is the line above which an account becomes an alert on somebody’s list. Move that line and two numbers move in opposite directions. Precision asks: of the accounts we flagged, how many were really leaving? Recall asks: of the accounts really leaving, how many did we flag? Precision is the cost paid by Maya’s team, in wasted calls to healthy customers. Recall is the cost paid by the business, in customers who walk out without ever appearing on a list.

Laid out as four boxes — the confusion matrix, an ugly name for a simple grid — every account falls into one of four fates. True alarms: flagged, and genuinely leaving. False alarms: flagged, and perfectly fine. Missed leavers: not flagged, and gone in eight weeks. And correctly left alone: not flagged, not going anywhere, which is most of the 400 and the box nobody celebrates.

There is no best setting. Lower the line and you catch more leavers, at the price of burning a six-person team’s week on companies like Hale & Porter LLP, quiet every August and renewing early regardless. Raise it and the team’s time is protected while real leavers — a Bramford Logistics whose champion has just left — slide out unflagged. Anyone who tells you a threshold can be optimised has quietly decided, on your behalf, what a wasted phone call and a lost $24k contract are each worth.

So Priya does not pick the threshold from the model’s scores. She picks it from the operation, in a room with Maya in it. The arithmetic starts at the wrong end on purpose: Maya has six people, they have other work, and a good intervention is a prepared call with the history read and an offer ready — not a templated email. Six people can do roughly ten to sixteen of those in a week. That is the constraint, and everything else is fitted to it.

At Northwind’s churn rate, about fourteen of the 400 accounts are genuinely on their way out in any ten-week window. The line producing sixteen alerts a week gives precision of roughly two in five and recall a little over forty per cent: sixteen accounts flagged, six of them genuinely leaving, ten calls to customers who were fine, eight leavers missed, 376 correctly left alone. Priya says all of that out loud, including the eight misses, because the alternative is somebody discovering them in month six and concluding the model was mis-sold.

The rejected alternatives show the shape of the trade. Set the line high and you get five alerts a week at three-in-five precision — a tidy list that ignores four fifths of the people leaving. Set it low and you get sixty alerts catching eight leavers in ten, which is excellent and entirely theoretical, because a team of six will work the first fifteen and stop reading. A list nobody can finish is not higher recall. It is the same recall with more guilt.

Where this goes wrong

Here is how it actually happens. The leaked feature stays in, because nobody asks where the number came from and 0.97 makes the fortnightly update a pleasure. The demo is a triumph, and expectations reset upward: the board hears “97% accurate” and Omar’s $250k envelope suddenly looks like a bargain. Then the pilot starts, the model is asked to predict a future that has not happened yet, and it performs like a coin toss with a good vocabulary. Nobody can explain the gap, because it was created five weeks earlier by a column nobody looked at closely.

What the organisation concludes is not “we validated badly”. It is “AI doesn’t work here”, and that sentence outlives everyone involved by about five years. The quieter version: a perfectly sound model handed over with the threshold left at 0.5 because 0.5 is the default, producing eighty alerts a week for a team of six, and dying of being ignored.

What Priya hands on

The deliverable is a model card and a ranked risk list. The card states what the model predicts, which features it uses, why two candidates were excluded and on what grounds, the honest precision and recall at the chosen threshold, and the known failure modes — trained on a period ending before the price rise, never having seen a March-style shock, certain to degrade. The list is the operational half: accounts above the line, ranked, each with the reason behind its score in Maya’s vocabulary. A note attached mentions that Sales has already asked for access to prioritise upsells.

It goes to Elena in module 9 with a deliberately provocative message: tell me what we are not allowed to do with this, before someone asks me to do exactly that. Elena audits those honest metrics, and honest metrics are the only kind that survive an audit.

Three constraints travel downstream. The threshold literally becomes the length of Jade’s Monday list in module 11 — sixteen names, not eighty and not five. The features chosen decide what the reason beside each alert can say, and the reason is what makes a human able to act: “12 of 50 seats used, two failed payments” sends a rep into a Juniper Health conversation about a smaller contract, where a bare 78% risk score sends them in with nothing. And the stated metrics decide what Nadia measures against in module 12, because a claim nobody wrote down cannot be tested later.

Spot the trap

Read each situation, decide what Priya would say, then tap a card to check.