Full-Time Senior Data Engineer

IFS is hiring a remote Full-Time Senior Data Engineer. The career level for this job opening is Expert and is accepting Sri Lanka based applicants remotely. Read complete job description before applying.

This job was posted 1 month ago and is likely no longer active. We encourage you to explore more recent opportunities on our site. However, you may still try your luck using 'Apply Now' link below. We recommend focusing on newer listings available here.

IFS

Job Title

Senior Data Engineer

Posted

Career Level

Full-Time

Career Level

Expert

Locations Accepted

Sri Lanka

Job Details

  • Design, develop, and maintain a generic ingestion framework capable of processing various types of data (structured, semi-structured, unstructured) from customer sources.
  • Implement and optimize ETL (Extract, Transform, Load) pipelines to ensure data integrity, quality, and reliability as it flows into the centralized datastore like Elasticsearch.
  • Ensure the ingestion framework is scalable, secure, efficient and capable of handling large volumes of data in real-time or batch processes.
  • Continuously monitor and enhance the data ingestion process to improve performance, reduce latency, and handle new data sources and formats.
  • Develop automated testing and monitoring tools to ensure the framework operates smoothly and can quickly adapt to changes in data sources or requirements.
  • Provide documentation, support, and training to other team members and stakeholders on using the ingestion framework.
  • Implement large-scale near real-time streaming data processing pipelines.
  • Design, support and continuously enhance the project code base, continuous integration pipeline, etc.
  • Build analytics tools that utilize the data pipeline to provide actionable insights into key business performance metrics.
  • Perform POCs and evaluate different technologies and continue to improve the overall architecture.
  • Experience building and optimizing Big Data data pipelines, architectures and data sets.
  • Strong proficiency in Elasticsearch, its architecture and optimal querying of data.
  • Strong analytic skills related to working with unstructured datasets.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data systems.
  • One plus years of experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems.
  • Candidates must have 4 to 6 years of experience in a Data Engineer role with  Bachelors or Masters (preferred) in Computer Science or Information Systems or equivalent field. Candidate should have knowledge of using following technologies/tools:
    • Experience working on Big Data processing systems like Hadoop, Spark, Spark Streaming, or Flink Streaming.
    • Experience with SQL systems like Snowflake or Redshift
    • Direct, hands-on experience in two or more of these integration technologies; Java/Python, React, Golang, SQL, NoSQL (Mongo), Restful API.
    • Versed in Agile, APIs, Microservices, Containerization etc.
    • Experience with CI/CD pipeline running on GitHub, Jenkins, Docker, EKS.
    • Knowledge of at least one distributed datastores like MongoDb, DynamoDB, HBase.
    • Experience using batch scheduling frameworks like Airflow (preferred), Luigi, Azkaban etc is a plus.
    • Experience with AWS cloud services: EC2, S3, DynamoDB, Elasticsearch

FAQs

What is the last date for applying to the job?

The deadline to apply for Full-Time Senior Data Engineer at IFS is 6th of December 2024 . We consider jobs older than one month to have expired.

Which countries are accepted for this remote job?

This job accepts [ Sri Lanka ] applicants. .

Related Jobs You May Like

Lead Data Scientist

USA
3 days ago
Data Analysis
Data Engineering
Machine Learning
Stellar Health
Full-Time
Experienced
YEAR $175000 - $220000

Data Scientist

USA
3 days ago
Data Analysis
Machine Learning
Python
Wpromote
Full-Time
Experienced
YEAR $117500 - $132500

Solutions Engineer

USA
4 days ago
Data Analytics
Data Visualization
Machine Learning
NobleAI
Full-Time
Experienced
YEAR $200000 - $220000

Field Application Scientist I

West Coast / Remote
4 days ago
Customer Support
Data Analysis
Instrumentation
LGC Group
Full-Time
Entry Level

Data Scientist II

USA
4 days ago
Data Analysis
Machine Learning
Python
Oscar
Full-Time
Experienced
YEAR $131200 - $172200

Senior Data Scientist

USA
5 days ago
data modeling
Lead Scoring
Python
Goodleap
Full-Time
Senior Manager

Healthcare Data Scientist

Ann Arbor, MI
5 days ago
Communication
Problem-solving
Python
ArborMetrix
Full-Time
Experienced
YEAR $75000 - $95000

Healthcare Data Scientist - SAS Programmer

Ann Arbor, MI
5 days ago
Data Analysis
Python
SAS
ArborMetrix
Full-Time
Experienced
YEAR $85000 - $100000

Senior Data Scientist

Costa Mesa, CA
6 days ago
Cloud Computing
Data Science
Machine Learning
Experian
Full-Time
Senior Manager

Senior Data Scientist

UK
1 week ago
Algorithms
Coding
Data Analysis
StackAdapt
Full-Time
Senior Manager

Staff Data Scientist

USA
1 week ago
Data Analysis
Healthcare Data
Machine Learning
Machinify
Full-Time
Expert
YEAR $200000 - $250000

Staff Data Scientist, Guest Journey

USA
1 week ago
Causal Inference
Data Analysis
Machine Learning
Airbnb
Full-Time
Expert
YEAR $192000 - $243000

Looking for a specific job?