Skip to content
Back to work

AI & Automation · Technology & SaaS

Customer Churn Prediction Model

For a subscription software business, a churn model that flags drifting accounts early enough for the customer success team to act, with the reasons behind every score.

Role
Product Owner & ML Solution Architect
Built on
Python, scikit-learn, pandas, CRM integration
Industry
Technology & SaaS
Project type
AI & Automation
Primary service
AI & Automation
Scope
Feature engineering, model development, evaluation, deployment and monitoring design.

Repository not shown as per company policy

  • 0

    signal families: product usage, support history and billing

  • 0

    churn definition, agreed before any model was trained

  • 0%

    of scores delivered with their contributing factors

  • 0

    new tools for the customer success team to learn

01Challenge

Subscription businesses learn about churn from the cancellation report.

By the time an account cancels, the relationship is over and the only remaining lever is a discount. Everything useful happens earlier, in signals nobody was consolidating.

  • Signals sat in separate systems

    Product usage, support history and billing state each told part of the story and none of them talked to each other.

  • No agreed definition

    Different teams counted churn differently, so a model trained on one definition would have been immediately disputed by another.

  • Leakage is easy and invisible

    Features that are only populated once an account is already leaving make a model look excellent in evaluation and useless in production.

  • Adoption is the real risk

    A score delivered in a new tool nobody opens changes nothing, however accurate it is.

02Approach

Treat it as an operational problem, not a modeling exercise.

  • Agree the target before the model

    One churn definition and one prediction horizon, signed off by the teams who would be measured against them.

  • Guard against leakage deliberately

    Features are computed as of the prediction point, so nothing that only exists after the decision can leak into training.

  • Deliver into existing tooling

    Scores land in the systems the customer success team already lives in, which is the difference between a model used and a model admired.

  • Plan for the model getting stale

    Retraining, versioning and performance tracking were designed in, because a model that is not monitored is a model that is quietly wrong.

03Outcome

A prediction nobody acts on is a report, not a product.

The modeling was the straightforward part. The work that made it useful was deciding who would act on a score, how much notice they needed, and what they could realistically do with it.

  • Define churn first

    Cancellation, downgrade and silent lapse are different events with different interventions. Agreeing one definition came before any feature engineering.

  • Build backwards from the action

    The notice period the team needs determines the prediction horizon, which determines which features are even allowed to be used.

  • Explain every score

    A risk number with no reasoning attached gets ignored by the third week. Contributing factors ship with the score.

What changed

  • Customer success teams receive a ranked list of at-risk accounts ahead of renewal, instead of learning about churn from the cancellation report.
  • Every risk score arrives with the factors that drove it, written in business language rather than feature names.
  • Drift and model quality are tracked after deployment, with scheduled retraining instead of a model left to go stale.

How it works

From raw events to an action a person takes.

scores and factorsretrainingProduct usageFeature adoption, frequency,depth of useSupport historyVolume, severity, sentiment,resolution timeBilling and planPlan changes, paymentevents, tenureCustomer success teamActs on scored accountsin the tools it already usesScored account listScore and the factorsbehind itSegmented viewsBy plan, tenure,account ownerMonitoringDrift and qualityafter deploymentFeature pipelinePoint-in-time correct, to prevent leakage1 Aggregation windowsRolling usage, support and billing features2 Point-in-time joinsComputed as of the prediction dateFeature validationNulls, ranges and drift checkedModelVersioned, reproducible runsTrainingTemporal validationEvaluationHeld-out and temporal splitsExplainabilityPer-account factors

One run, step by step.

01 / 05

Inside the system

What it is made of, and what keeps it safe.

What makes the score usable.

  • 1

    Feature pipeline

    Rolling aggregations over usage, support and billing data, joined point-in-time so every training row reflects only what was known at that moment.

    GuardrailLeakage checks on every feature. Anything only populated post-decision is excluded by construction, not by review.

  • 2

    Training and evaluation

    Versioned training runs with temporal validation, so performance is measured the way the model will actually be used.

    GuardrailA random split would flatter the model. Evaluation is always forward in time.

  • 3

    Explanation layer

    Each score ships with the factors that drove it, expressed in business language rather than feature names.

    GuardrailA score without reasoning is not shown. Unexplainable output is treated as a defect.

  • 4

    Delivery integration

    Scores and factors pushed into the tools the customer success team already uses, segmented by plan, tenure and owner.

    GuardrailNo new interface to adopt, which is what determines whether any of this gets used.

  • 5

    Monitoring and retraining

    Scheduled retraining with performance tracking and drift detection on both features and predictions.

    GuardrailDegradation surfaces as an alert rather than as a gradually ignored dashboard.

Built with

What it runs on.

Modeling
Pythonscikit-learnpandasNumPy
Pipeline
Scheduled feature jobsPoint-in-time joinsValidation checks
Delivery
CRM and success tooling integrationSegmented scored lists
Operations
Versioned training runsDrift monitoringScheduled retraining
Services appliedAI & AutomationData & AnalyticsDevOps & MLOps

Roadmap

Where the capability goes next.

Model

  • Uplift modeling, to predict who responds to an intervention rather than who is at risk
  • Survival modeling for time-to-churn rather than a fixed horizon

Operations

  • Automated champion and challenger comparison before promotion
  • Feature store so definitions are shared with other models

Product

  • Recommended next action alongside each risk score
  • Outcome tracking to close the loop between intervention and result

Advance warning, with the reasoning attached, in the tools the team already uses.

More work

Related projects.

View all work
AI & AutomationHealthcare

Autonomous AI Log Monitoring & Observability Platform

For a healthcare media and clinician engagement platform, a five-agent system that reads a production error, writes the fix and opens a reviewed pull request, with an engineer still deciding what ships.

AI & AutomationHealthcare

Drug Competitor Identification

A brand-intelligence tool that asks a language model who a drug competes with, then checks the answer against regulatory reference data before anyone is asked to trust it.

AI & AutomationHealthcare

AI Agents Platform

Seven agents that turn one upload into a recorded, print-ready batch of personalized posters, with a reviewer approving anything that carries commercial risk.

Discuss a similar project.

If something here is close to what you need, tell us about your situation and we will walk you through how we would approach it.

Discuss a Similar Projectcontact@genclover.com

Intelligence → Innovation → Automation → Growth