Azure Databricks Engineer
Clicking "Apply for this Position" redirects you to Capgemini's official careers portal
As an Azure Databricks Engineer, you will design and develop end-to-end data pipelines using Azure Data Factory, Databricks, PySpark, SQL, Delta Lake, and Azure Data Lake. You will build and optimize ETL workflows for ingestion, transformation, and loading of structured and semi-structured data into Azure Synapse Analytics and ADLS. Your responsibilities include implementing data validation and quality frameworks to ensure data reliability, writing efficient PySpark and SQL code for complex transformations, aggregations, and performance tuning. You will manage secure access using Azure Key Vault, role-based permissions, and compliance best practices. The role requires working in an Agile/Scrum environment and delivering solutions aligned with business needs. Candidates should have experience with Azure Data components such as Azure SQL Database, Azure SQL Warehouse, Synapse Analytics, and building CI/CD pipelines in data environments.