استخدام Senior/Mid-level Data Engineer

Who we are: Our team is cross-functional with a mix of Data analyst, BI developers and data engineers. We are working with open-source tools to help the business to make perfect decisions and also learning and developing ourselves.

Who you are: First off, we believe that you are a software engineer who can harness the complexity and volume in data whilst saving resources of the Data stack.

Secondly, you were the best of Programming, DB, DS, Algorithm and OS classes. You are a geek and invincible developer who has a passion to learn modern and open-source Big Data engines.

Why you should apply: Snapptrip has both mature and expanding products that are all data driven. In BI/Data team we are working with the leading modern open-source technologies to collect the products(Hotel, Flight, Train, Bus) data and subsequently, extract knowledge, So you have the opportunity to work directly with a bunch of cutting-edge open-source toolkits and distributed frameworks.

You will be doing:

• Developing and deploying applications in Scala, Spark(Scala or Python API) and Python to ingest/consume/analyze data in batch or Streaming manner
• Proposing solutions for some problematic issues and maintaining the whole Data stack includes PostgreSQL, HDFS, Kafka, Airflow, Metabase, Spark, Kubernetes and PrestoSQL (You will be central for the whole Data stack)
• You will support other roles to work easily with open source tools and resolve their performance issues in SQL and Spark

Core Qualifications:

• Bachelor's or a Master’s degree in Computer engineering (Software | AI)
• At least three years of practical experience in programming Scala, Java, or Python (preferably a JVM based language)
• Linux
• At least two years of practical experience in developing Spark applications
• Experience working with relational databases (preferably PostgreSQL)
• HDFS
• SQL
• Git
• Experience with CI tools (CI/CD or Jenkins)
• Prometheus/Grafana
• A thorough understanding of parallel and distributed computing (We run Spark applications deployed on Kubernetes cluster and process data on HDFS)
• A self starter
• Effective communication

Good to have:

• Kubernetes
• PrestoSQL (Trino)