Full-Time Staff Engineer Software Development

Job description

We seek a highly skilled Staff Engineer with extensive AWS and Big Data technologies expertise to join our Engineering team. The ideal candidate will have a proven track record in designing, developing, and implementing scalable software solutions leveraging AWS cloud services and Big Data platforms. This role requires strong programming skills, deep knowledge of AWS services, and experience with various Big Data tools and frameworks. In this role, you will be responsible for designing, implementing, and maintaining scalable, high-performance data solutions. You will lead complex projects, mentor junior engineers, and drive best practices for data engineering and cloud infrastructure.

 Responsibilities:

• Architect and Design: Design and implement scalable data solutions using AWS services (e.g., AWS EMR, AWS Athena, AWS Dynamo DB, AWS Glue, AWS S3) and Big Data technologies (e.g., Hadoop, Spark, Presto).
• Collaboration: Collaborate with cross-functional teams to gather requirements, define architecture, and implement solutions that meet business needs.
• Cloud Infrastructure: Manage and optimize AWS infrastructure to ensure high availability, performance, and cost efficiency. Utilize AWS tools for monitoring, security, and automation.
• Data Pipeline Development: Build and optimize data pipelines to handle large-scale data ingestion, processing, and storage. Ensure data quality and reliability
• Cloud Infrastructure: Manage and optimize AWS infrastructure to ensure high availability, performance, and cost efficiency. Utilize AWS tools for monitoring, security, and automation
• Leadership and Mentoring: Provide technical leadership and mentorship to junior engineers. Lead code reviews, and design reviews, and contribute to the development of engineering best practices
• Performance Optimization: Identify and resolve performance bottlenecks. Implement best practices for data storage, retrieval, and processing
• Innovation: Stay current with emerging technologies and industry trends. Propose and implement new tools, technologies, and processes to enhance data engineering practices
• Documentation and Reporting: Create and maintain comprehensive documentation for data architectures, processes, and systems. Provide regular updates and reports to stakeholders
• Develop Batch applications using Spark & Scala or PySpark
• Design, develop, and maintain complex data pipelines and workflows using Apache Airflow.
• Configure and manage Airflow environments, including scheduling, monitoring, and troubleshooting workflow execution.
• Implement monitoring, logging, and alerting solutions to ensure system reliability and availability.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• Proven experience as a Software Engineer, with at least 15 years of experience in developing distributed cloud-based applications
• Strong proficiency in programming languages such as Java, Python, or Scala, and familiarity with modern software development practices and tools
• AWS certifications such as AWS Certified Solutions Architect or AWS Certified Developer
• Experience with Big Data frameworks like Hadoop, Spark, Kafka and Athena as well as related tools such as Hive, HBase, and Presto

Preferred Qualifications: 

• AWS certifications such as AWS Certified Solutions Architect or AWS Certified Developer.
• Experience with AWS EMR, Lambda, ECS, Dynamo DB, Apache Airflow.
•  Familiarity with DevOps practices and tools for continuous integration, deployment, and automation.
• Knowledge of machine learning concepts and frameworks, and experience in deploying ML models on AWS infrastructure.
• Knowledge with stream processing technologies and real-time data analytics platforms. Join our team and play a key role in building cutting-edge software solutions that leverage the power of AWS and Big Data to drive innovation and business growth.