Full-Time Lead Data Engineer
Wiser Solutions is hiring a remote Full-Time Lead Data Engineer. The career level for this job opening is Manager and is accepting Vadodara, India based applicants remotely. Read complete job description before applying.
Wiser Solutions
Job Title
Posted
Career Level
Career Level
Locations Accepted
Share
Job Details
When looking to buy a product, it can be hard to find one with desired characteristics at a suitable price, or it might be out of stock. Brands/retailers also struggle to ensure a smooth product selection experience. Wiser aims to provide this seamless experience by improving visibility for brands and retailers. Our goal is to solve this efficiently and cost-effectively.
We collect, categorize, and analyze structured and semi-structured data from various sources (e.g., millions of products from hundreds of websites, data from thousands of stores). We help customers discover patterns to improve competitiveness, both online and in-store.
We are seeking a lead-level software engineer to lead a team developing data architecture for data collection and analytics. If you are passionate about optimization, scaling, and integration, this role might be for you.
What You Will Do
- Collaborate with product and engineering leaders to define data solutions supporting customer practices.
- Design/develop/extend data pipeline services and architecture.
- Mentor junior team members and share expertise with others.
- Ensure high quality data standards.
- Take responsibility for shepherding projects from idea to production.
- Bring new ideas to the table.
Technologies We Use
- Languages: SQL, Python
- Infrastructure: AWS, Docker, Kubernetes, Apache Airflow, Apache Spark, Apache Kafka
- Databases: Snowflake, Trino/Starburst, Redshift, MongoDB, Postgres, MySQL
- Others: Tableau
Requirements
- 10+ years of professional software engineering experience
- Strong proficiency with data languages (Python, SQL)
- Strong proficiency with data processing technologies (Spark, Flink, Airflow)
- Strong proficiency with RDMS/NoSQL/Big Data solutions
- Solid understanding of streaming solutions (Kafka, Pulsar)
- Solid understanding of Docker, Kubernetes, ETL/ELT, OLTP/OLAP concepts
- Solid understanding of data structures (Parquet, ORC, Avro)
- Solid understanding of Apache Iceberg
- Proven ability to transform data into structured format
- Solid understanding of AWS, Linux, and infrastructure
- Proven ability to diagnose and address data abnormalities
- Experience building data warehouses and data lakes
- Experience with business intelligence solutions (Tableau)
- Understanding of Domain Driven Design and Microservice Architecture
- Passion for data, analytics, or machine learning
Bonus Points
- Experience with vector databases
- Retail/ecommerce experience
- Proficiency in other programming languages (Scala, Java, Golang)
- Experience with Apache Arrow