استخدام Senior Data Engineer

About Snapp

Snapp is the pioneer provider of ride-hailing mobile solutions in Iran that connects smartphone owners in need of a ride to drivers who use their private cars, offering transportation services. We are ambitious, passionate, engaged, and excited about pushing the boundaries of the transportation industry to new frontiers and being the first choice of each user in Iran.

About the Role

The role contributes to a variety of exciting projects ranging from designing robust and automated Data Pipelines and Storage processes to building tools for improving company-wide productivity with data. It’s more about designing, implementing, and operating stable, scalable, and efficient solutions to flow data from different sources into a data lake and other databases. You will work as a stakeholder to bring the data into standard/query-able format and empower the company to take data-driven decisions. Our Team empowers nearly all of Snapp Cab/Box to take data-driven decisions and make an impact throughout the company.

Responsibilities

• Develop and automate large scale, High-Performance, Scalable data pipelines (batch and Streaming) to drive faster analytics
• Ability to Design new Data Architecture with excellent run-time characteristics such as low latency, fault-tolerance, and availability
• Maintain and monitor Real-Time Analytics and Big Data Systems to make sure about their reliability and resolve issues
• Collaborating with Business Intelligence team, Data Scientists team, Ventures and other teams to build data insights and help them to achieve their business goals
• Design, implement and maintain scalable real-time and batch data pipelines handling billions of records
• Maintain Real-Time Analytics Systems and Big Data Systems and make sure their reliability and maintenances
• Setup Real-Time Analytics solution depending on services
• Propose new data Architecture for new requirements and fine-tune the existing ones
• Monitoring data services and resolving issues in case of any incident

Basic Requirements

• BS/MS or more in computer engineering/science or related experience
• At least 2 years of programming experience in Python, Java or Scala, or Go
• SQL Knowledge and Experience with Database systems (Click house, MySQL, Postgres, and other DBs)
• Specialized in Hadoop ecosystem (HDFS, Yarn, Hive, Spark)
• Hands-on experience with Kafka, Zookeeper, Logstash Preferred Requirements
• Experience working with one or more of these: Airflow, Debezium, Confluent Schema Registry
• Familiar with Monitoring systems like Grafana, Prometheus, Exporters
• Experience in Streaming technologies like Spark, Apache Flink, Nifi
• Hands-on experience in Linux, Docker, Kubernetes, and Virtualization
• Experience with data exploration and data visualization like Hue, Superset
• Experienced in Agile, Scrum, DevOps projects
• Good Communication and Teamwork Skills