Full-Time Senior Data Lake Developer

Job Title: Senior Data Lake Developer

Job Description:

Our client is building a new data lake solution, "Origin," which will serve business intelligence, data science, and other initiatives within CIB.

Key Features:

• Near real-time data ingestion from multiple sources.
• Data storage in AWS cloud with decoupled storage/compute.
• Data compression and partitioning using Parquet format.
• Centralized metadata catalogue.
• Integration with AWS services (Athena, EMR, QuickSight) and third-party solutions (Apache Spark, Presto, Tableau).

Team & Project:

The project is sponsored at high levels and involves a small, agile development team focused on high-quality solutions for electronic trading and pricing.

Responsibilities:

• Deliver high-quality software meeting scope and quality expectations.
• Own technical design and ensure comprehensive documentation of the system architecture.
• Collaborate with partner technology teams.

Required Experience:

• 5-10 years of data team experience.
• Strong understanding of big data ingestion and large datasets.
• Expertise in creating and maintaining high-quality ETLs.
• Deep understanding of Tableau.
• Proficient in SQL, Hive, and Hadoop.
• Experience with Python, Pyspark, Pandas, and JupyterLab.
• Experience with AWS platforms.
• Knowledge of continuous integration and continuous delivery tools (Git, Jenkins).
• Experience with agile development and software life cycle.
• Excellent English communication skills (B2+).

Nice to Have:

• Kafka experience.
• EMR (Elastic Map Reduce) experience for Spark clusters.
• Terraform and Ansible experience.
• Bash scripting, Boto3 experience.
• Continuous integration and continuous delivery tool configuration experience.

Soft Skills:

• Energetic, motivated, and results-oriented.
• Adaptable to diverse technical responsibilities.
• Strong analytical and problem-solving skills.
• Excellent time management and communication skills.

Benefits:

• Full-time contract
• Training and career development opportunities
• Multicultural team and international projects

What is the last date for applying to the job?

Which countries are accepted for this remote job?