Educational requirements: Bachelor
English requirements: Competent English
Requirements for skilled employment experience for years: 3-5 years
Required residence status: Temporary visa, Permanent resident, Citizen
Accept remote work: unacceptable
About the Opportunity
We are looking for an experienced Data Engineer with a focus on Spark to work on a range of projects for our clients.
You'll need to have at least five years' corporate experience in data engineering using Spark to drive commercial business outcomes. The ideal candidate's favourite words are process, data, scale, and agile. You will leverage your strong collaboration skills and the ability to streamline the components and processes in a big data environment
Responsibilities:
Build robust, efficient and reliable big data pipelines from different sources for machine learning applications Build codes using the existing standards, lifecycle management framework and version control Design and executive unit tests to ensure data compliance Identify various data risks associated with the data process and draft mitigation strategies Extract knowledge and insights from structured or unstructured data Work closely with the business users and support them on building their Alteryx workflow to Spark Development on Spark (using Java or Scala) – documenting requirements and code testing
Qualifications:
At least 5 years' of experience in a similar data engineering role within a corporate environment Extensive hands on experience using Apache Spark Strong programming skills in particular SQL, Spark SQL, R, Python, Scala and Java AWS experience Good business acumen to interpret complex technical data and present results to business stakeholders Experience working with different technical and non-technical stakeholders Strong interpersonal and communications skills with ability to interact with the business autonomously Good knowledge of Hadoop ecosystem (including HDFS, Spark, Oozie, and Hive) Bachelor Degree (Preferred) Excellent written & verbal communication skills
