Thiết kế và vận hành pipeline dữ liệu batch và real-time, DataMart (Customer Profile, Journey, Event) cho nhóm CRM trên Azure và Google Cloud.
Trách nhiệm
Xây dựng pipeline ETL/ELT bằng SQL/Python; vận hành CDC Debezium tới Azure Databricks
Thiết kế DataMart theo kiến trúc Medallion/Data Vault; làm việc với CDP/CRM
Quản lý, giám sát tài nguyên và chi phí trên Azure, Google Cloud; xử lý sự cố pipeline
Kiểm tra chất lượng dữ liệu tự động, IAM, mã hóa và CI/CD qua Git, Azure DevOps
Tóm tắt bằng AI từ JD gốc, đọc JD gốc trước khi ứng tuyển.
JOB DESCRIPTION
1. Primary Objective:
Design and build scalable batch and real-time ETL/ELT data pipelines
Design data models and DataMarts (Customer Profile, Journey, Event) per business requirements
Operate, monitor, and optimize the data platform on Microsoft Azure and Google Cloud
Integrate source systems and provide technical support to other teams in the Data Department
Ensure data quality and adherence to data governance standards across pipelines
Provide technical guidance, code review, and knowledge sharing within the team
2. Main Responsibilities
Build and optimize ETL/ELT pipelines using SQL/Python from trading, CRM, and APIs; directly operate CDC flow via Debezium on GKE Autopilot to Azure Databricks
Design DataMarts using Medallion/Data Vault architecture; collaborate with stakeholders and vendors to clarify requirements and integrate CDP/CRM
Manage, operate, and monitor resources/costs on Azure (Databricks, ADF, Synapse/Fabric, ADLS) and Google Cloud (GKE Autopilot, Pub/Sub)
Monitor pipelines, troubleshoot incidents, automate alerting, and continuously improve system performance
Implement automated data quality checks, reconciliation, and defect handling (using Great Expectations or equivalent)
Integrate core trading, CRM, and accounting source systems; lead data migrations/upgrades with zero data loss or downtime
Implement IAM access control, encryption, and data masking; perform backup/recovery and establish CI/CD workflows via Git and Azure DevOps
Support Analytics, Data Science, and Governance teams in data usage; mentor Data Engineers and review their code
3. Other Responsibilities
Maintain technical documentation, research new data technologies, and handle ad-hoc data requests
Perform other tasks as assigned by the Data Engineering Manager
JOB REQUIREMENTS
1. Educational level
Bachelor’s degree in information technology, Computer Science, Data Science or a related field
2. Professional Experience
5+ years of experience as a Data Engineer with strong ETL/ELT pipeline development and large-scale data modeling
Advanced SQL proficiency and hands-on experience with cloud data platforms (Azure Databricks, BigQuery, Redshift, Synapse, Snowflake, Spark)
Familiarity with CDP/CRM ecosystems and data integration with marketing platforms (Facebook Ads, Google Analytics, Criteo, RTB House)
Practical experience with workflow orchestration and transformation tools (dbt, Airflow, Azure Data Factory)
Strong Python skills for data processing, automation, and API integration (Scala/Java for Spark is a plus)
Experience with real-time event streaming technologies (Kafka, Pub/Sub, Event Hub)
Knowledge of Data Governance (data quality, lineage, catalog) and frameworks like Great Expectations or Soda
Experience with CI/CD for data engineering (Git, GitHub Actions, Azure DevOps) and exposure to Docker/Kubernetes
Prior involvement in enterprise-scale CDP/CRM integration projects
