Full-Time Data Engineer

Job Overview

The Data Engineer will be responsible for designing and building modern data platforms to support data-driven decision making. The Data Engineer will execute technical implementation of data engineering and visualization projects.

The Data Engineer will help build a data and analytics consulting practice by taking part in recruiting efforts, creating technical collateral, and staying on top of technology trends with ongoing training and certifications. The entire consulting team will be responsible for building long-term strategic relationships with clients and participating in all aspects of project delivery

Key Responsibilities

• Design, implement, and develop data pipelines to collect and process large amounts of data from various sources.
• Implement data storage solutions that are scalable, secure, and efficient, such as data warehouses and databases.
• Develop and implement data validation and testing processes to ensure that data is processed accurately and efficiently.
• Automate data collection, processing, and reporting processes to minimize manual work and improve efficiency.
• Create high quality documents to capture problem statements, requirements, solutions and designs.
• Contribute to the development of reusable, repeatable collateral for use across the practice.
• Implement data quality checks and validation processes to ensure the accuracy, consistency, and security of data.
• Monitor data pipelines and processes, troubleshoot issues, and implement solutions to prevent recurrence
• Maintain comprehensive documentation of data architectures, processes, and workflows to facilitate knowledge sharing and collaboration.

Key Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 3-5 years of experience in data management, data engineering, or data visualization.
• Excellent problem-solving, organization, debugging, and analytical skills.
• Ability to work independently and in a team environment.
• Excellent communication skills for effectively expressing ideas to team members and clients.
• Understanding of relational database concepts and SQL querying.
• Knowledge of cloud computing platforms, such as Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure
• Experience building data pipelines with ETL tools/Equivalent Cloud services, such as Azure Data Factory, dbt, AWS Glue, Matillion or equivalent.
• Familiarity with data warehousing solutions, such as Amazon Redshift, Snowflake, Google BigQuery, or Azure Synapse.
• Experience with visualization tools, such as Power BI, Looker, Tableau, and QuickSight.
• Debug and optimize existing data infrastructure and processes as needed.

Nice To Have

• Experience building large-scale, high throughput, 24x7 data systems.
• Any data engineering, visualization, or data science certifications on any of the clouds.
• Experience with at least one programming language, such as Python, Java, or Vue.
• Exposure to machine learning algorithms, AI, and/or LLM, with implementation in practice.
• Experience with legacy data systems (e.g. Hadoop, Informatica).
• Experience in Migrating Data Engineering workloads from on-prem to cloud or between clouds
• Experience in GCP Data services is preferred