The Customer
Our client is one of Europe’s established financial institutions, operating across retail, corporate, and investment banking with a large and complex data ecosystem. Behind every transaction, every report, and every regulatory filing sits years of layered data infrastructure — one that had, over time, become deeply entangled with a legacy Oracle Data Warehouse and SAS processing environment.
Data was central to their operations. But parts of the platform it ran on had become a liability.
The Challenge
By 2025, the bank’s data engineering teams were facing a problem familiar to many large enterprises: the infrastructure supporting key analytical and reporting workflows was holding them back.
Specific Oracle DWH databases — those responsible for feeding group-level reporting — had accumulated years of dependencies, with applications delivering data through tightly coupled pipelines. Licensing costs were rising. Scalability was constrained. And as the rest of the data world moved toward cloud-native, open platforms, these legacy components remained an island.
The question wasn’t whether to modernize — it was how to do it without disrupting critical reporting operations
“The dependencies were deep. Every application had its own quirks, its own data types, its own ingestion patterns. We couldn’t just lift and shift — we had to rebuild with precision.”
The Journey
The bank’s data teams evaluated migration paths carefully. Simply replacing one tool with another wasn’t enough — they needed an architecture that would scale, support modern data governance, and give engineering teams flexibility for years to come.
The decision: migrate to Databricks using a Medallion Architecture — a layered approach where raw data lands in a Bronze Layer, is cleansed and structured in a Silver Layer, and made available for consumption downstream.
But choosing the destination was the easy part. The real challenge was execution.
The Discovery
To execute a migration of this scale — touching live banking applications while maintaining data integrity — the bank needed experienced data engineers who could hit the ground running. They turned to IDS Consulting, a specialist in data warehousing and outsourced data expertise.
IDS Consulting embedded a dedicated team of senior data engineers directly into the bank’s department, working alongside internal teams as a seamless extension of their workforce.
The Solution
IDS Consulting designed and executed the migration from Oracle DWH to Databricks — layer by layer, application by application. The approach followed the Medallion Architecture:
- Bronze Layer: Raw data ingested from source systems, preserving full history and lineage
- Silver Layer: Cleansed, standardized, and business-rule-applied data — ready for downstream consumption
The migration targeted the decommissioning of the Oracle DWH databases used for group reporting, eliminating associated Oracle and SAS licensing costs while aligning these critical pipelines with the bank’s long-term Databricks strategy.
The Implementation
The project launched in Q3 2025, with an estimated completion in 2027 — a deliberate, phased approach that prioritized data integrity over speed. The IDS Consulting team encountered — and solved — two significant technical challenges:
Challenge 1 — XML Ingestion Gap
Two business-critical applications were delivering data in XML format. Databricks, at the time of migration, lacked native XML ingestion support. Rather than blocking progress or forcing a risky direct-source rewrite, the team engineered a pragmatic solution: pull the data from DWH History instead of directly from source. This preserved data continuity while keeping the migration on track.
Challenge 2 — Data Type & Metadata Mapping
One source system presented complex data type mismatches between Oracle and Databricks schemas. The team developed a custom data type mapping layer, handling edge cases in metadata translation to ensure no data was lost or miscast during migration.
Both solutions reflect IDS Consulting’s core approach: deliver pragmatic, production-grade solutions under real-world constraints — not just theoretical architectures.
The Results
The migration is actively in progress, with early results already validating the investment:
- Licensing cost reduction: Oracle and SAS dependencies tied to group reporting pipelines are being systematically decommissioned, eliminating significant recurring licensing overhead
- Improved scalability: Databricks’ cloud-native architecture removes the capacity ceilings that constrained the legacy platform
- Modernized data pipeline: Group reporting flows are now aligned with the bank’s strategic Databricks platform — enabling faster analytics, better data governance, and reduced technical debt
- Zero data loss: Despite complex XML and metadata challenges, the migration has maintained full data integrity across all migrated applications
“By 2027, the bank will have fully migrated its group reporting data pipelines off Oracle and SAS — replacing legacy complexity with a scalable, cost-efficient, and future-ready data platform.”