Phát triển REST/gRPC API để expose dịch vụ inference và tích hợp với hệ thống nội bộ
Xây dựng pipeline CI/CD, monitoring, alerting, logging, metrics, distributed tracing cho production AI services
Có thì tốt
Kinh nghiệm production system với SLA rõ ràng
Kiến thức chuẩn hóa model format: ONNX, TensorRT engines
Lương và phúc lợi
Bảo hiểm xã hội, bảo hiểm y tế theo luật
Gói FPT Premium Care
Tài trợ khóa học và chứng chỉ liên quan
Môi trường làm việc sáng tạo, cởi mở, tôn trọng cá nhân
Tóm tắt bằng AI từ JD gốc, đọc JD gốc trước khi ứng tuyển.
Job Objective
Design, implement, and optimize AI model serving systems, particularly for Large Language Models (LLMs), ensuring optimal performance, reliability, and cost efficiency.
Key Responsibilities
Design and implement backend and serving engines for AI models using frameworks such as vLLM, NVIDIA TensorRT-LLM, KServe, TorchServe, TensorFlow Serving, or equivalent technologies.
Develop lazy loading and on-demand loading mechanisms for model layers or modules to optimize memory footprint and reduce initialization time.
Optimize inference performance through batching, caching, quantization, model parallelism, GPU/CPU utilization optimization, and system profiling.
Design and develop REST/gRPC APIs to expose inference services and integrate them with internal systems, gateways, and client applications.
Build deployment pipelines and implement monitoring and alerting systems for production AI services, including logging, metrics, and distributed tracing.
Collaborate with Research/Applied AI teams to facilitate model handoff and standardize model formats such as ONNX, TensorRT engines, and other relevant formats.
Job Requirements
Hands-on experience deploying AI model backend/serving engines using at least one of the following technologies: vLLM, NVIDIA TensorRT-LLM, KServe, TorchServe, TensorFlow Serving, or equivalent. Candidates with experience deploying production systems with clearly defined SLAs are preferred.
Strong understanding of deep learning model architectures, particularly Transformer-based models, and lazy loading mechanisms for model layers/modules (e.g., partial model loading, sharded weights, memory mapping, and CPU/GPU offloading).
Proficiency in at least one programming language, preferably Python, along with experience using web frameworks such as FastAPI, Flask, or Django to build inference services.
Solid knowledge of backend systems, including HTTP, basic networking, asynchronous programming, message queues, caching (Redis/in-memory cache), and microservices architecture.
Hands-on experience with Docker, container orchestration (Kubernetes preferred), and CI/CD for AI services.
Understanding of monitoring and observability tools such as Prometheus, Grafana, OpenTelemetry, or equivalent technologies.
Top Benefits
Salary: competitive, paid according to ability; negotiated during the interview.
Social insurance and health insurance according to labor laws.
Creative, open-minded working environment that respects individuals.
FPT Premium Care package.
Activities and culture with FCI and FPT Corporation.
Sponsorship for related courses and certifications.
Own a production DaaS/VDI platform end-to-end, serving thousands of virtual desktops.
Direct influence on the FCD product roadmap, working closely with Product and customers.
Working Environment
Working location: 7th Floor, FPT Tower, no. 10 Pham Van Bach Street, Dich Vong ward, Cau Giay district, Hanoi.
Working hours:
Full time
8h30 AM – 12h00 PM
1h00 PM – 5h30 PM
Contact Person
Pham Thi Ha My (Ms.) – Talent Acquisition Team Leader
Email: [email protected]
Phone: 0962456194
FPT Smart Cloud (FCI) Co., LTD
Address: 7th Floor, FPT Tower, No. 10 Pham Van Bach, Cau Giay Dist, Hanoi
Websites: FPT Cloud | FPT AI