Thiết kế, xây dựng và tối ưu các pipeline dữ liệu từ Kafka vào data warehouse.
Yêu cầu
5+ năm kinh nghiệm Data Engineering, gồm 3+ năm dùng dbt và Airflow production
Từng thiết kế và làm chủ cấu phần platform dữ liệu hoặc sản phẩm data hoàn chỉnh
Từng giữ vai trò chính trong việc chuyển đổi từ data warehouse cũ sang mới
Thành thạo data modeling, dbt Core, SQL, Postgres, Python, Kafka, GitOps và K8s
Giao tiếp tiếng Anh tự tin cả nói và viết
Trách nhiệm
Xây dựng và vận hành pipeline từ Kafka và CSDL nguồn vào kho dữ liệu bằng Airflow, dbt
Thiết kế mô hình dữ liệu cho báo cáo và triển khai bằng dbt project dễ bảo trì
Di chuyển dữ liệu, logic nghiệp vụ và báo cáo từ kho dữ liệu cũ sang nền tảng mới
Giám sát pipeline và chất lượng dữ liệu hằng ngày, xử lý sự cố tận gốc
Có thì tốt
Kinh nghiệm ngành ngân hàng, fintech, thanh toán hoặc hiểu quy định SBV
Kinh nghiệm với Spark, Grafana, Prometheus
Hiểu biết về quản trị dữ liệu (phân loại, masking, audit truy cập)
Tóm tắt bằng AI từ JD gốc, đọc JD gốc trước khi ứng tuyển.
Work directly with business users to gather requirements, clarify business logic and propose suitable data solutions.
Design data models for reporting: define grain, metrics and dimensions, and implement them as maintainable dbt projects.
Build and own streaming and batch pipelines from Kafka and source databases into the warehouse and data marts, using Airflow, dbt, and Postgres.
Migrate data, business logic and reports from the legacy data warehouse to the new platform, including backfill, parallel run, reconciliation, cutover and rollback.
Deliver new data features and products end to end, from design to deployment and handover.
Deploy pipelines and services through GitLab CI following GitOps practices; keep all changes version-controlled and reviewed.
Monitor pipelines and data quality daily (tests, freshness, reconciliation, alerts); troubleshoot incidents and fix root causes.
Tune performance of SQL queries, dbt models and Airflow jobs.
Join on-call rotation for critical data products.
Maintain documentation: data models, lineage, runbooks, architecture diagrams.
Mentor team members through code review, design review and pairing; set and maintain team standards for modeling, dbt and CI/CD.
5+ years in data engineering, including 3+ years running dbt and Airflow in production.
Has designed and owned at least one data platform component or data product end to end.
Has led or played a key role in at least one migration from a legacy data warehouse to a new platform.
Experience working with on-premise or self-hosted data platforms (not only managed cloud services)
Technical expertise (must have)
Data modeling (advanced): dimensional modeling (Kimball, star/snowflake schema, SCD types, fact table design, grain definition). Designs models that stay stable as business requirements change.
Dbt Core (advanced): project structure (staging/intermediate/marts), incremental models, snapshots, macros and Jinja, packages, tests and documentation; orchestrates dbt runs with Airflow and sets up CI in GitLab (e.g. slim CI using manifest/state artifacts); optimizes model performance and run time
GitOps & CI/CD (advanced): branching and merge request workflow, automated testing and environment promotion in GitLab CI; declarative deployments on Kubernetes (Helm, ArgoCD or equivalent).
Data migration (advanced): assesses and re-implements legacy transformation logic, backfills historical data, runs old and new systems in parallel, reconciles results, and plans cutover and rollback.
SQL & Postgres (advanced): writes complex analytical SQL (window functions, CTEs, incremental logic); reads query plans and tunes indexes, partitioning and queries for reporting workloads.
Kafka (proficient): creates and configures topics; builds producers and consumers in production; understands partitioning, consumer groups, offsets and schema management; handles late, duplicate or out-of-order events.
Python (proficient): writes production-quality code for pipelines and tooling (structured, tested, reusable); builds custom Airflow operators and hooks.
Kubernetes (proficient): deploys and runs data workloads as pods: resource sizing, config and secrets, reading logs and troubleshooting failures. Cluster administration not required.
Soft skills
Takes ownership of problems through to resolution, not just the assigned task.
Communicates clearly with non-technical business users; able to challenge requirements and explain trade-offs.
Willing and able to mentor less experienced engineers.
English: communicates confidently in both speaking and writing with foreign colleagues, including technical discussions and documentation.
Nice to have
Banking, fintech or payments domain; knowledge of SBV regulations.
Spark.
Grafana, Prometheus.
Data governance: classification, masking, access audit.
Education
Bachelor's degree in IT, Computer Science or related fields (preferred).
