Senior Data Engineer (CRM)

JOB DESCRIPTION

1. Primary Objective:

• Design and build scalable batch and real-time ETL/ELT data pipelines
• Design data models and DataMarts (Customer Profile, Journey, Event) per business requirements
• Operate, monitor, and optimize the data platform on Microsoft Azure and Google Cloud
• Integrate source systems and provide technical support to other teams in the Data Department
• Ensure data quality and adherence to data governance standards across pipelines
• Provide technical guidance, code review, and knowledge sharing within the team

2. Main Responsibilities

• Build and optimize ETL/ELT pipelines using SQL/Python from trading, CRM, and APIs; directly operate CDC flow via Debezium on GKE Autopilot to Azure Databricks
• Design DataMarts using Medallion/Data Vault architecture; collaborate with stakeholders and vendors to clarify requirements and integrate CDP/CRM
• Manage, operate, and monitor resources/costs on Azure (Databricks, ADF, Synapse/Fabric, ADLS) and Google Cloud (GKE Autopilot, Pub/Sub)
• Monitor pipelines, troubleshoot incidents, automate alerting, and continuously improve system performance
• Implement automated data quality checks, reconciliation, and defect handling (using Great Expectations or equivalent)
• Integrate core trading, CRM, and accounting source systems; lead data migrations/upgrades with zero data loss or downtime
• Implement IAM access control, encryption, and data masking; perform backup/recovery and establish CI/CD workflows via Git and Azure DevOps
• Support Analytics, Data Science, and Governance teams in data usage; mentor Data Engineers and review their code

3. Other Responsibilities

• Maintain technical documentation, research new data technologies, and handle ad-hoc data requests
• Perform other tasks as assigned by the Data Engineering Manager

JOB REQUIREMENTS

1. Educational level

• Bachelor’s degree in information technology, Computer Science, Data Science or a related field

2. Professional Experience

• 5+ years of experience as a Data Engineer with strong ETL/ELT pipeline development and large-scale data modeling
• Advanced SQL proficiency and hands-on experience with cloud data platforms (Azure Databricks, BigQuery, Redshift, Synapse, Snowflake, Spark)
• Familiarity with CDP/CRM ecosystems and data integration with marketing platforms (Facebook Ads, Google Analytics, Criteo, RTB House)
• Practical experience with workflow orchestration and transformation tools (dbt, Airflow, Azure Data Factory)
• Strong Python skills for data processing, automation, and API integration (Scala/Java for Spark is a plus)
• Experience with real-time event streaming technologies (Kafka, Pub/Sub, Event Hub)
• Knowledge of Data Governance (data quality, lineage, catalog) and frameworks like Great Expectations or Soda
• Experience with CI/CD for data engineering (Git, GitHub Actions, Azure DevOps) and exposure to Docker/Kubernetes
• Prior involvement in enterprise-scale CDP/CRM integration projects

3. Professional Competencies/Knowledge

• Data Classification and Archiving
• Data Management Process Understanding
• Data Analysis Knowledge
• CRM System Knowledge
• Data Architecture Knowledge
• Database Management Knowledge
• Data Recovery Knowledge
• Information Technology Knowledge
• Project Management Knowledge