Senior Data Engineer (Python, ETL)

Mô tả công việc

Job Summary: 

For one of our clients, we are seeking an experienced Data Engineer to join our dynamic team. The ideal candidate will have a solid background in data engineering, with expertise in Python, ETL tools, Airflow, and Hadoop. You will be responsible for building, maintaining, and optimizing data pipelines that handle high-volume transactions exceeding 50 million per day. 

If you're excited about solving challenging problems, delivering polished user experiences, and contributing to a fast-paced and innovative environment, we’d love to meet you. 

Key Responsibilities: 

• Design, develop, and maintain scalable data pipelines and data models for large transaction systems. 
• Utilize Python and ETL tools to extract, transform, and load data efficiently from various sources. 
• Implement and manage workflows using Apache Airflow to automate data processing tasks. 
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality and availability. 
• Optimize data processing performance and reliability in a Hadoop environment. ● Monitor and troubleshoot data pipeline issues to ensure operational efficiency and reliability. 
• Document processes, data flow, and architecture for future reference and compliance. ● Stay current with industry trends and emerging technologies in data engineering.