Senior Data Engineer

Job Description Roles & Responsibilities As part of strengthening our Data teams, we are looking for profiles capable of designing, industrializing and optimizing data platforms (batch & real-time) within distributed environments based on Cloudera. Your responsibilities: Development & Industrialization Develop massive data processing pipelines in PySpark (Batch and Real-Time / Streaming modes). Set up real-time streams via Kafka (topics, partitions, schemes, offsets) and ensure ingestion with NiFi. Model and optimize NoSQL schemas, especially on Cassandra (tables, keys, clustering, replication) and Hive. Integrate and transform data from multiple sources (APIs, databases, streams, files). Quality, Performance & Reliability Deploy Data Quality mechanisms (controls, monitoring, alerting). Optimize Spark processing (partitioning, tuning, data formats) specifically for distributed architectures. To ensure the supervision and resolution of incidents in a production environment. CI/CD & Governance Industrialize developments via CI/CD chains (automated testing, deployments). Documenting flows, models and best practices. Collaborating occasionally with third-party teams on Dataviz topics and collaborating with Data Scientists (particularly on customer segmentation algorithms). Contribute to data governance (catalogue, traceability, security). Desired Candidate Profile Experience: ~4 years of experience in distributed environments and Big Data architectures. Architecture: Strong interest and significant experience in On-Premise infrastructures. Spark / PySpark: Essential mastery in Batch and Streaming processing. NoSQL: Confirmed experience on Cassandra and/or Hive Streaming & Ingestion: Very good mastery of Kafka and NiFi Nice-to-have / Bonus: Good understanding of the Hadoop/HDFS ecosystem. Cloudera proficiency is a plus but not required. Knowledge of Data Science (clustering/customer segmentation models) and Data Visualization tools. Tools & DevOps: Skills in Git, CI/CD (GitLab CI) and an orchestration tool (Airflow, Luigi, Prefect). Company Industry IT - Software Services Department / Functional Area IT Software Keywords Senior Data Engineer Get real-time job updates only on our App