Chapter Lead - Data Engineering

The role of Chapter Lead – Data Engineering is to drive consistent and production grade data engineering practices and frameworks across Optus data platforms and lead Data engineering teams across multiple agile squads. The roleis responsible for development and automation of Big Data and Cloud ingestion and transformation pipelines and data consumption services; and ensuring modern operations to deliver consumer driven Big Data and Cloud enabled solutions.

This role manages inter-dependencies with other ongoing data initiatives and manage demand/supply for Big Data and Cloud engineering resources in chapter.


The role

  • Implement request for data ingestion, transformation, and preparation of data sources in an Agile squad
  • Develop and execute jobs to import data periodically/ (near) real-time from an external source
  • Setup a streaming data source to ingest data into the platform
  • Deliver data sourcing approach and data sets for analysis, with activities including data staging, ETL, data quality, and archiving
  • Profile source data and validate fit-for-purpose
  • Works with Delivery lead and Solution Architect to agree pragmatic means of data provision to support use cases
  • Manage data engineering activities in a squad and support product owner in key decisions
  • Maintain relationships and manage inter dependencies across other data analytics squads across Agile Tribes
  • Manage talent acquisition and development of chapter members in modern technologies

 

The perks

 

We offer all kinds of benefits, such as:

  • Work collaboratively in an open, agile environment with flexible working hours and location
  • Discounts with over 400 companies Australia wide (Technology, Retail, Home, Fashion and more!)
  • Mobile and Broadband staff discounts
  • Onsite facilities at Macquarie Park such as a Gym, GP, Mini-Mart, Cafes
  • Training, Mentoring and further learning opportunities
  • Staff busses to Epping and Wynyard, and back again

 

About you

Preferred skills and experience include:

  • Bachelor’s degree in maths, statistics, computer science, information management, finance or economics
  • 8-15 years of hands on advanced experience in Big data and Cloud engineering solutions and leadership of large-scale engineering teams.
  • Deep understanding of business and analytics objectives.
  • Ability to drive design and development of Data Lake standards.
  • Experience in Lamba and Kappa design frameworks leveraging Hadoop batch and streaming components
  • Experience in design and build of data lake/hub environments in large telco organisations
  • Experience in ingestion technologies (e.g. sqoop, nifi, flume), processing technologies (Spark/Scala) and storage (e.g. HDFS, HBase, Hive)
  • Experience in data profiling, source-target mappings, ETL development, SQL optimisation, testing and implementation
  • Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential
  • Experience managing structured and unstructured data types
  • Experience in requirements engineering, solution architecture, design, and development / deployment
  • Experience in creating big data or analytics IT solution
  • Track record of implementing databases and data access middleware and high-volume batch and (near) real-time processing
  • Agile Ways of Working experience (preferably in the capacity of a Chapter Lead across Agile Tribes)
  • Strong value realisation skills, including understanding the financial implications of initiating or foregoing decisions
  • Effective communicator – able to present content and gain buy-in from stakeholders
  • Able to lead Data engineering teams across multiple squads and address conflicting priorities

 

You would be a self-starter with the ability to work independently and multitask several different activities across critical deadlines in a high-pressure environment.

 

About us

 

At Optus, we don’t sit back and let the future happen to us – we’re out there making it.  By expanding into new technology and relentlessly improving every day, we’re working to create a better tomorrow for all Australians.

 

Recover Password

No worries! Enter the email used for registration and we’ll send you a confirmation link.

Get started as talent

Welcome , what describes you best?

This could be your current role, or where you want to work in the future.

What industries would you like to follow

Get on the radar of the companies doing amazing things in:

What’s your location?

Would you like to be contacted with relevant opportunities?

Skip talking to agency recruiters, skip the job adverts and let employers contact you directly.

What are your skills

Add your CV

OR

Microsoft Word .doc and PDF accepted

Email address

Set a password for your account

Minimum 6 characters long.

By signing up, you agree to our Terms of Service and Privacy Policy.