Legacy Oracle DWH to Cloud-Native Lakehouse

IDS designed a cloud-native lakehouse on Azure — Synapse for the warehouse layer, Data Factory for orchestration, Databricks for the transformation logic previously buried in PL/SQL — replacing monolithic batch jobs with modular, testable pipelines.

Client

A mid-size commercial bank

Services provided

About the project

Our solution

The Customer: Who they are

A commercial bank running its core data warehouse on Oracle for over 15 years, supporting risk, finance, and regulatory reporting.

The Challenge: Identifying the villain

Batch windows had crept from 4 hours to 11, ETL logic lived in undocumented PL/SQL packages written by people who’d since left the bank, and every new regulatory report meant weeks of archaeology before a single line of new code. “We were maintaining a museum, not a data platform,” said the Head of Data Engineering  [placeholder].

The Journey: The search for a solution

The bank evaluated a straight “lift and shift” to cloud infrastructure, but a proof of concept showed it would simply move the same brittle logic to more expensive infrastructure — no batch window improvement, no maintainability gain.

The Discovery: Finding IDS

IDS was brought in through a technology partner recommendation, specifically for replatforming experience that went beyond infrastructure migration into re-architecting the pipelines themselves.

The Solution: The hero arrives

IDS designed a cloud-native lakehouse on Azure — Synapse for the warehouse layer, Data Factory for orchestration, Databricks for the transformation logic previously buried in PL/SQL — replacing monolithic batch jobs with modular, testable pipelines.

The Implementation: The battle

Migration was phased by subject area, running the legacy and new pipelines in parallel with automated reconciliation until each domain matched exactly. The trickiest obstacle was a set of undocumented risk calculations with no clear business owner; IDS worked with risk analysts to reverse-engineer and validate the logic before cutover.

The Results: The happy ending

  • Batch window: 11 hours → 2.5 hours
  • Infrastructure cost: 30% reduction post-migration
  • Time to build a new regulatory report: weeks → days
  • Zero reporting downtime during cutover

“We can finally add a new data source without holding our breath.” — Head of Data Engineering 

Technologies used: Azure Synapse · Azure Data Factory · Databricks · Oracle (legacy source)

Facebook
LinkedIn
Send on E-mail

More use cases

One Platform, Two Data Worlds: A Big Data Analytics Mart for Banking Group Controlling

Client: Leading Banking Group
IDS designed and built a new analytical data mart purpose-built for Controlling, constructed entirely on big data technologies rather than the client’s Oracle stack.
Read more

Migrating Without Missing a Beat: A New Data Warehouse Foundation for a Banking Controlling Team

Client: Leading Banking Institution
When the migration to the new DWH began, IDS took on the task of carrying that Datamart — and the business logic behind it — onto the new platform.
Read more

From Hadoop to Databricks: Scaling a Banking Data Platform Beyond Its Limits

Client: Banking Client
IDS Consulting was brought in to provide hands-on technical support and consultancy for the migration to Databricks.
Read more

From Days to Minutes: An 8-Year Data Warehouse Transformation for a Leading Telecom Operator

Client: Leading Telecom Operator
IDS Consulting was brought in not for a single fix, but as an ongoing extension of the client’s Data Warehouse team. The mandate: modernize how data moved through the platform, and keep it evolving safely as the business — and the underlying infrastructure — kept changing around it.
Read more

Full-Stack Data Platform with Embedded Testing & Knowledge Transfer

Client: A mid-size IT services company
IDS delivered the platform end-to-end (Python and .NET services, automated test coverage from day one) while running a structured knowledge transfer track in parallel — pairing IDS engineers with the client’s internal team throughout, not just in a handover week at the end.
Read more

A Full-Stack Commerce Data Platform, Built and Tested End-to-End

Client: A Full-Stack Commerce Data Platform, Built and Tested End-to-End
IDS delivered full-stack development (Java backend, React front end) with a dedicated QA workstream running in parallel from the start — functional, regression, and performance testing built into every sprint — under a single programme manager accountable for scope, timeline, and quality together.
Read more
Contact