Educational requirements: Bachelor
English requirements: Competent English
Requirements for skilled employment experience for years: 1-3 years
Required residence status: Temporary visa
Accept remote work: Accept during COVID-19 restrictions only
As a Data Engineer you should embrace the challenge of dealing with big data on a daily basis (Kafka, RDS, Redshift, S3, Athena), perform best practices of ETL/ELT, and build tools for proper data ingestion from multiple data sources. You'll collaborate closely with data engineers, data analysts and Software Engineers across different teams, find bottlenecks and solve the problem. Design, implement and maintain the heterogeneous data processing platform to automate the execution and management of data-related jobs and pipelines.
Implement automated data workflow in collaboration with data analysts, continue to improve, maintain and improve systems in line with growth, terabytes of data Daily.
Implement different procedures to ensure data quality and data confidence. Collaborate with Software Engineers on application events, and ensure right data can be extracted.
Contribute to resource management for computation and capacity planning. Diving deep into code and constantly innovating
Perform best practices of ETL/ELT, and build tools for proper data ingestion from multiple data sources.
What you'll need
Experience with AWS data technologies (EC2, EMR, S3, Redshift, EKS, Glue) and infrastructure.
Working knowledge in big data frameworks such as Apache Spark, Apache Kafka, Apache Airflow.Rich experience with Linux and database systems (Postgres)
Experience with relational and NoSQL database, query optimisation, and data modelling
Experience in one or more of the following: Scala/Java, SQL, Python, ShellExperience with container technologies (Docker, k8s), Agile development, DevOps (Terraform) and CI tools.Excellent problem solving skillsExcellent verbal and written communication skillsAbility of developing end to end data solutions, with quality and monitoring.
Demonstrated ability to bridge between business and data analytics
Intellectual curiosity and an understanding of what drives customer outcomes and has a positive impact on the business
