Full-Time Big Data Engineer

In this role, you will work with our Business Analytics & Insights (BAI) team, collaborating with cross-functional teams to deliver high-quality data solutions in areas such as Supply Chain, Finance, Operations, Customer Experience, HR, Risk Management, and Global IT.

Responsibilities:

• Lead the technical planning for data migration, including data ingestion, transformation, storage, and access control within Azure Data Factory and Azure Data Lake.
•  Design and implement scalable, efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks.
•  Develop reusable frameworks for the ingestion of large datasets.
•  Ensure data quality and integrity by implementing robust validation and cleansing mechanisms throughout the data pipeline.
•  Work with event-based/streaming technologies to ingest and process data in real-time.
•  Provide technical support to the team, resolving challenges during the migration and post-migration phases.
•  Stay current with the latest advancements in cloud computing, data engineering, and analytics technologies; recommend best practices and industry standards for data lake solutions.

• 4+ years of IT experience.
•  Minimum of 4 years working with Azure Databricks.
•  Proficiency in Data Modeling and Source System Analysis.
•  Strong knowledge of PySpark and SQL.
•  Experience with Azure components: Data Factory, Data Lake, SQL Data Warehouse (DW), and Azure SQL.
•  Experience with Python for data engineering purposes.
•  Ability to conduct data profiling, cataloging, and mapping for technical design and construction of data flows.
• Familiarity with data visualization/exploration tools.

• Technical leadership skills with the ability to guide and mentor teams.
• Strong problem-solving capabilities, able to translate business requirements into data science solutions.
•  Excellent communication skills, with the ability to explain complex concepts to both technical and non-technical stakeholders.
• Strong team player with excellent interpersonal and collaboration skills.
• Ability to deliver high-quality results within specified timelines.

What is the last date for applying to the job?

Which countries are accepted for this remote job?