Expert Data Architect Databricks

Job Description Roles & Responsibilities Job Description: Within the Technology Department, the BI/Data team manages and executes Business Intelligence and Data Engineering projects, ensuring the quality, performance, and scalability of solutions. You will play a key role in the design, optimization, and monitoring of our data pipelines. You will contribute your Databricks expertise to improve the architecture, performance, and reliability of our data flows for internal clients. Main Responsibilities: As a Databricks expert, you will be involved in the main aspects of designing, optimizing, and evolving the data platform: Design, develop, and optimize data processing and transformation pipelines on Azure Databricks and Azure Data Lake. Define and implement robust and scalable data architectures to support the growth of data volumes. Implement automated testing and ensure the quality and reliability of data pipelines. Participate in the definition and optimization of CI/CD pipelines via Azure DevOps. Apply best practices in Data Engineering and Software Engineering at all stages of development. Collaborate with Data Product Owners and business teams to propose innovative solutions tailored to needs. Supervise data processing and ensure monitoring via Datadog or equivalent tools. Identify and implement optimizations to improve performance and reduce costs on Databricks. Main technical environment: Languages: SQL, PySpark, Python, Scala. Tools & BI: ​​Power BI, SSIS, SSAS, Business Objects, Datadog, Snyk. Cloud & Data: Azure Databricks, Azure Data Factory, ADLS Gen2. CI/CD & DevOps: Git, Azure DevOps. Desired Candidate Profile Your strengths to make a difference: Master's degree (Bac+5) in computer science, engineering, or a related field. Minimum 5 years of experience in Data Engineering, Data Architecture, or Databricks, with experience leading teams or projects. Advanced proficiency in several languages: Python, PySpark, Scala, SQL. Proven expertise in developing and optimizing Databricks pipelines, with the ability to define technical standards and best practices. Proficiency in Spark UI for diagnosing and improving the performance of complex processes. Very good knowledge of the Azure Cloud ecosystem and its data-oriented services (ADF, ADLS, Databricks). Experience implementing best development practices: clean code, version control, unit testing, CI/CD, and BI modeling principles. Advanced experience with dbt or similar frameworks for SQL transformation and data orchestration. Experience working in an Agile methodology, collaborating with Data Product Owners and project teams. Leadership and teamwork skills: able to guide and coach team members, make strategic decisions, and propose innovative solutions. Ability to adapt quickly to a constantly evolving technical environment and proactive in suggesting improvements to architecture and processes. Company Industry IT - Software Services Department / Functional Area IT Software Keywords Expert Data Architect Databricks Get real-time job updates only on our App