Full-Time Lead Data Engineer
Wiser Solutions is hiring a remote Full-Time Lead Data Engineer. The career level for this job opening is Manager and is accepting Vadodara, India based applicants remotely. Read complete job description before applying.
Wiser Solutions
Job Title
Posted
Career Level
Career Level
Locations Accepted
Share
Job Details
When looking to buy a product, it can be hard to find the right one at the right price. Finding it in stock can be frustrating.
Brands and retailers want to provide the best shopping experience. We at Wiser believe in seamless shopping experiences, providing brands and retailers with visibility.
Our goal is to solve problems elegantly and cost-effectively. We collect, categorize, and analyze structured and semi-structured data from various sources (e.g., 20M+ products from 500+ websites, data from 300,000 brick-and-mortar stores).
We help customers be more competitive by discovering data patterns for use across online and in-store sales.
We are seeking a lead-level software engineer to lead a team in developing data architecture for data collection and an analytics platform.
What You Will Do
- Define data solutions supporting customer business practices.
- Design, develop, and extend data pipeline services and architecture.
- Mentor junior team members and share expertise.
- Ensure high quality data standards.
- Own projects from idea to production.
- Bring new ideas to the table.
Technologies We Use
- Languages: SQL, Python
- Infrastructure: AWS, Docker, Kubernetes, Apache Airflow, Apache Spark, Apache Kafka, Terraform
- Databases: Snowflake, Trino/Starburst, Redshift, MongoDB, Postgres, MySQL
- Others: Tableau
Requirements
- Bachelor's/Master's in Computer Science or related field
- 10+ years of professional software engineering experience
- Strong proficiency with SQL and Python
- Proficiency with Spark, Flink, and Airflow
- Experience with RDMS/NoSQL/Big Data solutions
- Understanding of streaming solutions (Kafka, Pulsar, Kinesis)
- Hands-on experience with Docker, Kubernetes, Terraform, Helm charts
- Understanding of ETL/ELT and OLTP/OLAP concepts
- Solid understanding of data structures (Parquet, ORC, Avro)
- Understanding of Apache Iceberg, or other open table formats
- Ability to transform raw data into structured data
- AWS, Linux, and infrastructure concepts
- Ability to diagnose and address data abnormalities
- Data warehouse experience using conformed dimensional models
- Data lake experience (Trino, Dremio, Druid)
- Experience with business intelligence tools (Tableau)
- Experience with ML/AI pipelines
- Understanding of Domain Driven Design and Microservice Architecture
Bonus Points
- Experience with vector databases
- Retail or eCommerce experience
- Proficiency in other languages (Scala, Java, Golang)
- Experience with Apache Arrow
Supervisory Responsibilities
- Mentorship on best practices
- Agile ceremonies leadership