Tóm tắt
Skills bắt buộc
MoMo's data platform processes petabytes of data through thousands of pipelines every day. It powers product analytics, business performance analysis, insight validation, and machine learning models for fraud detection, credit scoring, and personalization. Its reliability directly affects business decisions and product performance;We are moving from a platform built primarily on BigQuery toward a hybrid lakehouse across cloud and on-premises infrastructure. The shift addresses growing demands for data control, lower cost, and regulatory compliance. It also introduces harder operational problems: distributed compute, constrained capacity, platform migration, and the separation of workloads by data sensitivity;The Data Orchestration team owns data ingestion, movement between environments, and modeling for downstream users. You will build and operate pipelines and models as the platform moves to the lakehouse, keeping existing reports and data products dependable. Middle engineers own pipelines and models within the team's standards, with support on broader design decisions. Senior engineers set designs and standards, balance correctness, performance, and cost, and lead changes involving other teams;Mô tả công việcBring new data sources to the lakehouse. Build batch, CDC, and streaming pipelines from operational databases, backend services, Kafka, APIs, and partner feeds. Develop reusable ingestion components and frameworks so connecting new sources becomes repeatable and increasingly automated;Keep data accurate as sources change. Own data correctness from ingestion through transformation, with schema validation, data quality checks, and reconciliation against source data. Work with source owners to handle schema changes and prevent missing or duplicate records from affecting downstream users;Turn raw data into useful models. Build layered dbt models, from raw data through conformed entities to data marts for reporting, analytics, and data products. Work with Analytics and other data consumers to agree on business definitions and create models teams can reuse across BigQuery and the lakehouse;Scale processing at a sustainable cost. Improve Spark pipeline performance and resource efficiency as data volumes grow. Use production workload metrics to identify bottlenecks and balance runtime, reliability, and compute cost;Keep sensitive data protected. Work with Security and Data Infrastructure to ensure pipelines and models respect access controls and the separation of PII and PII-free workloads across environments;Own reliability in production. Keep Airflow workflows, pipelines, and data models dependable against agreed objectives for data freshness and availability. Use monitoring and alerts to detect failures, restore service, and prevent recurring incidents. Seniors set service objectives and guide decisions when problems span teams;Yêu cầu công việcMust have2+ years of experience in Data Engineering, Data Platform Engineering, or a related backend or infrastructure role, with production systems you built or ran;Strong SQL and hands-on experience with at least one of Python, Java, or Scala;Hands-on experience with at least one distributed data technology such as Apache Spark, Apache Flink, Kafka, BigQuery, or an MPP engine;Experience building and running scheduled or streaming pipelines in production, with Airflow or a similar orchestrator;A solid grasp of data fundamentals: data modeling, partitioning, incremental loads, and why a pipeline can produce a wrong result;Git, code review, and CI/CD as normal practice. Nice to haveExperience with dbt or data quality and testing frameworks;Experience with open table formats such as Apache Iceberg, Delta Lake, or Apache Hudi;Experience with CDC and streaming ingestion tools such as Debezium or Kafka Connect;Experience with Kubernetes, Docker, and infrastructure as code (Terraform, Pulumi, Helm);Experience with monitoring and SRE practice, such as Prometheus, Grafana, SLOs, and on-call;Experience with GCP, AWS, or hybrid cloud and on-premise environments;Experience with payment, fintech, or other high-volume transaction data;Experience with data governance, access control, or data privacy requirements;In your application, tell us which of the areas above you would most like to work on.
Đang tải…
Nguồn: careers MoMo
VN fintech super-app
HCMC · Hanoi · Sync · 122Tăng 17 job so với 3 ngày trước job
1 referrer sẵn sàng hỗ trợ
Bấm Xin referral trên job để kết nối người hỗ trợ.
122 job
Tóm tắt
Skills bắt buộc
MoMo's data platform processes petabytes of data through thousands of pipelines every day. It powers product analytics, business performance analysis, insight validation, and machine learning models for fraud detection, credit scoring, and personalization. Its reliability directly affects business decisions and product performance;We are moving from a platform built primarily on BigQuery toward a hybrid lakehouse across cloud and on-premises infrastructure. The shift addresses growing demands for data control, lower cost, and regulatory compliance. It also introduces harder operational problems: distributed compute, constrained capacity, platform migration, and the separation of workloads by data sensitivity;The Data Orchestration team owns data ingestion, movement between environments, and modeling for downstream users. You will build and operate pipelines and models as the platform moves to the lakehouse, keeping existing reports and data products dependable. Middle engineers own pipelines and models within the team's standards, with support on broader design decisions. Senior engineers set designs and standards, balance correctness, performance, and cost, and lead changes involving other teams;Mô tả công việcBring new data sources to the lakehouse. Build batch, CDC, and streaming pipelines from operational databases, backend services, Kafka, APIs, and partner feeds. Develop reusable ingestion components and frameworks so connecting new sources becomes repeatable and increasingly automated;Keep data accurate as sources change. Own data correctness from ingestion through transformation, with schema validation, data quality checks, and reconciliation against source data. Work with source owners to handle schema changes and prevent missing or duplicate records from affecting downstream users;Turn raw data into useful models. Build layered dbt models, from raw data through conformed entities to data marts for reporting, analytics, and data products. Work with Analytics and other data consumers to agree on business definitions and create models teams can reuse across BigQuery and the lakehouse;Scale processing at a sustainable cost. Improve Spark pipeline performance and resource efficiency as data volumes grow. Use production workload metrics to identify bottlenecks and balance runtime, reliability, and compute cost;Keep sensitive data protected. Work with Security and Data Infrastructure to ensure pipelines and models respect access controls and the separation of PII and PII-free workloads across environments;Own reliability in production. Keep Airflow workflows, pipelines, and data models dependable against agreed objectives for data freshness and availability. Use monitoring and alerts to detect failures, restore service, and prevent recurring incidents. Seniors set service objectives and guide decisions when problems span teams;Yêu cầu công việcMust have2+ years of experience in Data Engineering, Data Platform Engineering, or a related backend or infrastructure role, with production systems you built or ran;Strong SQL and hands-on experience with at least one of Python, Java, or Scala;Hands-on experience with at least one distributed data technology such as Apache Spark, Apache Flink, Kafka, BigQuery, or an MPP engine;Experience building and running scheduled or streaming pipelines in production, with Airflow or a similar orchestrator;A solid grasp of data fundamentals: data modeling, partitioning, incremental loads, and why a pipeline can produce a wrong result;Git, code review, and CI/CD as normal practice. Nice to haveExperience with dbt or data quality and testing frameworks;Experience with open table formats such as Apache Iceberg, Delta Lake, or Apache Hudi;Experience with CDC and streaming ingestion tools such as Debezium or Kafka Connect;Experience with Kubernetes, Docker, and infrastructure as code (Terraform, Pulumi, Helm);Experience with monitoring and SRE practice, such as Prometheus, Grafana, SLOs, and on-call;Experience with GCP, AWS, or hybrid cloud and on-premise environments;Experience with payment, fintech, or other high-volume transaction data;Experience with data governance, access control, or data privacy requirements;In your application, tell us which of the areas above you would most like to work on.
Đang tải…
Nguồn: careers MoMo
