Full-Time Senior Data Engineer

Senior Data Engineer

You will be a key member of our Data Engineering team, focused on designing, developing, and maintaining robust data solutions on on-prem environments.

You will work closely with internal teams and client stakeholders to build and optimize data pipelines and analytical tools using Python, Scala, SQL, Spark and Hadoop ecosystem technologies.

This role requires deep hands-on experience with big data technologies in traditional data centre environments (non-cloud).

Responsibilities

• Design, build, and maintain on-prem data pipelines to ingest, process, and transform large volumes of data from multiple sources into data warehouses and data lakes
• Develop and optimize Scala-Spark and SQL jobs for high-performance batch and real-time data processing
• Ensure the scalability, reliability, and performance of data infrastructure in an on-prem setup
• Collaborate with data scientists, analysts, and business teams to translate their data requirements into technical solutions
• Troubleshoot and resolve issues in data pipelines and data processing workflows
• Monitor, tune, and improve Hadoop clusters and data jobs for cost and resource efficiency
• Stay current with on-prem big data technology trends and suggest enhancements to improve data engineering capabilities

Requirements

• Bachelor's degree in software engineering, or a related field
• 5+ years of experience in data engineering or a related domain
• Strong programming skills in Python or Scala
• Expertise in SQL with a solid understanding of data warehousing concepts
• Hands-on experience with Hadoop ecosystem components (e.g., HDFS, Hive, Apache Hudi, Iceberg and Delta Lake)
• Proven ability to design and manage data solutions in on-prem environments (no cloud dependency)
• 3rd party data integrations from different sources (including APIs)
• Proficiency in Airflow or similar orchestration tool
• Strong problem-solving skills with an ability to work independently and collaboratively
• Excellent communication skills and ability to engage with technical and non-technical stakeholders

Good to Have

• Master’s degree in data science or related field
• Knowledge on Google and Facebook APIs and accessing S3 and SFTP buckets
• Prompt engineering with basic GenAI understanding
• Excellent written and verbal English

What is the last date for applying to the job?

Which countries are accepted for this remote job?