About the team:
We own the data pipelines for the entire department — we design, build, and operate them. We handle both batch and streaming workloads at game-data scale.
What you’ll do:
Design and build batch and streaming data pipelines.
Monitor pipelines, troubleshoot failures, and fix root causes.
Optimize Spark/SQL jobs and run backfills safely.
Ensure data quality, including audited financial data.
Use AI to automate repetitive work.
Must-have:
3+ years of experience in Data Engineering, including at least 1 year building and operating production pipelines.
Strong SQL and Python skills.
Hands-on experience with Spark, including performance tuning and troubleshooting OOM errors and data skew.
Experience with Airflow or a similar orchestration tool.
Understanding of the infrastructure around pipelines: storage, compute, and logging.
Proficiency with Git and code review workflows.
Regular use of AI tools in daily work, with concrete examples of automation.
Disciplined and detail-oriented; willing to respond to critical issues outside office hours.
Nice-to-have:
Experience with Kafka/CDC streaming.
Familiarity with Trino, Iceberg, Hive/HDFS, and S3.
Experience with Scala.
Experience working with audited or financial data.
Experience with game data.