Data Engineer MS Fabric PySpark

Job Description Roles & Responsibilities Your main tasks will be: Design, develop and maintain large-scale data pipelines using PySpark. Set up data flows with Azure Data Factory, integrating various data sources and destinations (SQL, Data Lakes, APIs, etc.). Ensure the integration of data from various systems in a cloud environment (Azure). Optimize the performance of data processing (SQL, PySpark, etc.). Collaborating with business teams, data analysts and data scientists to understand needs and propose effective and scalable solutions. Participate in defining the data architecture and governance strategy. Set up and monitor data processing jobs (error handling, performance management). Document the solutions implemented and follow best development practices. Participate in the continuous improvement of existing processes and tools. Desired Candidate Profile Formation et exp rience : Master's degree (Bac +5) in computer science, data science, or an equivalent field. Minimum of 3 years of experience in a similar role, with proven experience in development on PySpark and Azure Data Factory. strong>Comp tences techniques :/strong> Mastering PySpark for processing massive data in a distributed environment. Significant experience in setting up and managing Azure Data Factory for creating and orchestrating data pipelines. Solid experience in relational and non-relational databases (SQL, NoSQL). Knowledge of data management tools in the Azure ecosystem (Azure Data Lake, Azure SQL Database, Azure Blob Storage, etc.). Practical knowledge of Azure cloud services and their use in data processing solutions (Azure Databricks, Azure Synapse Analytics, etc.). Experience with version control tools (Git, Azure DevOps, etc.) and good software development practices. Knowledge of the concepts of Big Data and performance optimization in distributed environments. Personal skills: Autonomy, rigor and attention to detail. Excellent ability to solve complex technical problems. Good communication skills and ability to work in a team. Analytical mindset and ability to understand business needs in order to propose appropriate solutions. Company Industry IT - Software Services Department / Functional Area IT Software Keywords Data Engineer MS Fabric PySpark Get real-time job updates only on our App