The Customer: Who they are
A digital-first retail bank processing high transaction volumes with a data science team under pressure to reduce fraud losses without frustrating genuine customers.
The Challenge: Identifying the villain
The bank’s data science team had built a promising fraud model in notebooks, but it never made it to production — there was no MLOps infrastructure to deploy, monitor, or retrain it, and the underlying transaction data had quality issues the model quietly absorbed. “Our model worked great in the notebook and never once in the real world,” said the Head of AI/ML.
The Journey: The search for a solution
The team tried to hand the model to the engineering group for manual deployment, but without a repeatable pipeline, every retraining cycle became a bespoke project that ate weeks of engineering time.
The Discovery: Finding IDS
The bank’s CDO connected with IDS through the bank’s existing data governance engagement, recognizing the fraud model’s data quality problems traced back to the same underlying issues.
The Solution: The hero arrives
IDS rebuilt the fraud pipeline on governed transaction data, then operationalised the model using Databricks and MLflow for experiment tracking, versioning, and automated retraining, with Azure ML handling deployment and monitoring for drift.
The Implementation: The battle
The model was rolled out in shadow mode first — scoring live transactions without acting on them — so the fraud team could validate performance before it started blocking transactions. The main obstacle was false-positive tuning; IDS worked iteratively with the fraud operations team to calibrate thresholds against real cost of both fraud losses and customer friction.
The Results: The happy ending
- False positive rate: reduced by 35%
- Fraud caught pre-transaction: +25% vs. rules-based system
- Model retraining cycle: manual/ad hoc → automated monthly
- Time from model update to production: weeks → days [placeholder]
“This is the first model that actually survived contact with production.” — Head of AI/ML
Technologies used: Databricks · MLflow · Python · Scikit-learn · Azure ML