Full-Time Data Engineer - Open Finance Specialist
Experian is hiring a remote Full-Time Data Engineer - Open Finance Specialist. The career level for this job opening is Experienced and is accepting São Paulo, Brazil based applicants remotely. Read complete job description before applying.
Experian
Job Title
Posted
Career Level
Career Level
Locations Accepted
Share
Job Details
Our team is responsible for processing features for machine learning, which feeds the company's main products and score models. We work with petabyte-scale data volumes, processing the largest database in Latin America. We are looking for a dedicated and passionate technology professional to join us on this innovation journey. Your initial role will be in the Open Finance project, building data movement pipelines in the cloud and categorizing transactions, applying proprietary NLP tools to data processing, aiming to feed products and help millions of people improve their scores.
In this role, you will primarily work with Spark, running on EMR and orchestrated via Airflow. Be prepared to learn, as Serasa has a world of unique concepts and products. We are looking for someone who can quickly understand architectural patterns and propose performant solutions, helping us bring products to life.
Daily tasks include:
- Translation of code for Spark (Python and Scala)
- Building data pipelines
- Building Airflow Dags
- Optimizing EMR resource usage per job
- Code optimization
- Database validation
- Unit test coverage
- Project follow-up and defense in architecture committees
Skills:
- Python
- Spark
- Airflow
- AWS
- EMR
- Kafka (desirable)
- Scala (desirable)