Overview
We’re looking for a Databricks Engineer to join one of our Data Engineering teams. You’ll design, build, and optimize scalable data solutions using the Databricks Lakehouse Platform, enabling advanced analytics, Business Intelligence, and AI initiatives. You’ll work closely with Data Engineers, Data Scientists, BI Developers, Solution Architects, and business stakeholders to deliver reliable, high-performing, and cloud-native data platforms.
Requirements
Data engineering & solution design
• Design, develop, and maintain scalable data solutions using the Databricks Lakehouse Platform.
• Translate business and technical requirements into efficient data processing solutions.
• Design data pipelines supporting batch and streaming workloads.
• Collaborate with Solution Architects to define modern data platform architectures.
• Develop reusable frameworks, templates, and best practices for Databricks development.
• Produce and maintain technical documentation, data flow diagrams, and solution designs.
Data pipelines & data integration
- Build and optimize ETL/ELT pipelines using Apache Spark and Databricks.
- Develop scalable data ingestion processes from multiple structured and unstructured data sources.
- Implement Delta Lake architectures to ensure reliable and efficient data storage.
- Optimize data transformations for performance, scalability, and maintainability.
- Support incremental processing and Change Data Capture (CDC) patterns.
- Ensure data quality, consistency, and reliability across enterprise data pipelines.
Data modeling & analytics
- Design and implement analytical data models supporting reporting and advanced analytics.
- Collaborate with Data Modellers and Business Analysts to translate business requirements into scalable data structures.
- Develop curated data layers following modern Lakehouse architecture principles.
- Optimize datasets for Business Intelligence, Machine Learning, and self-service analytics.
- Support implementation of data governance, metadata, and lineage requirements.
- Contribute to enterprise data architecture standards and best practices.
Performance & optimization
- Optimize Spark jobs, notebooks, and Delta Lake tables for performance and cost efficiency.
- Analyze query execution plans and improve workload performance.
- Implement partitioning, caching, file optimization, and workload tuning strategies.
- Monitor cluster utilization and optimize compute resources.
- Troubleshoot production issues affecting data processing and platform performance.
- Continuously improve solution scalability and operational efficiency.
Testing, deployment & operations
- Develop automated testing and validation processes for data pipelines.
- Collaborate with DevOps and DataOps teams to implement CI/CD pipelines.
- Support deployment across development, testing, and production environments.
- Monitor production workloads and resolve operational issues.
- Maintain technical documentation throughout the solution lifecycle.
- Contribute to operational excellence and continuous improvement initiatives.
Desired profile
Must have
- University degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field.
- 3+ years of experience in Data Engineering or Big Data development.
- Hands-on experience with Databricks and Apache Spark.
- Strong programming skills in Python and SQL.
- Experience building ETL/ELT pipelines for enterprise data platforms.
- Good understanding of Delta Lake, Lakehouse architecture, and modern data engineering principles.
- Experience working with Git and CI/CD practices.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
Nice to have
- Experience with Azure Databricks, Microsoft Fabric, Azure Data Factory, Azure Synapse Analytics, or Snowflake.
- Familiarity with Spark Structured Streaming and real-time data processing.
- Experience with Unity Catalog, Delta Live Tables, or Databricks Workflows.
- Knowledge of Infrastructure as Code and cloud automation.
- Experience with Docker, Kubernetes, or DataOps practices.
- Experience in Banking, Retail, Insurance, or Telco industries.
- Databricks Certified Data Engineer Associate or Professional certification.
- Microsoft Azure certifications.
How you work
- Passionate about modern data engineering and cloud technologies.
- Strong ownership and attention to detail.
- Analytical, proactive, and solution-oriented.
- Comfortable collaborating with multidisciplinary teams.
- Focused on delivering scalable, reliable, and high-quality data solutions.
- Committed to continuous learning and technology innovation.
Interested and suitable for this job?
Apply now for Databricks Engineer position