About us

Job Description of : Lead AI Platform Engineer

Job Title : Manager / Sr. Manager

Department : Information Technology

Reports To : Assistant Vice President

Experience : 8 to 12 years

Preferred Qualification : BE/ B Tech/ MCA

Required Qualification : BE/ B Tech/ MCA

1) Deep expertise in AI Platform engineering, distributed systems, or AI(GPU) infrastructure set up on Premises and on Sovereign Cloud.
2) Hands-on experience in architecting and scaling production AI platforms and environments 
3) Experience on MLOps/LLMOps pipelines and distributed model inference.J2

Core Competencies :

1) 	Languages & Frameworks: Proficiency in Python for high-throughput network proxies and custom microservices. Strong foundation in asynchronous programming (FastAPI, AsyncIO, gRPC).
2) 	Inference Runtimes & Serving: Production expertise with vLLM, NVIDIA Triton Inference Server, TensorRT-LLM, Ray Serve, Hugging Face TGI, and llama.cpp.
3) 	API Gateway & Networking: Deep knowledge of API Gateway patterns, Envoy, Kong, LiteLLM, NGINX, Redis, WebSocket streaming, and Server-Sent Events (SSE).
4) 	Container & Cluster Orchestration: Expertise in Docker, Kubernetes, Helm, NVIDIA GPU Operator, KEDA (autoscaling), and Linux kernel networking on Private and Public cloud platforms
5) 	Observability & Guardrails Stack: Prometheus, Grafana, OpenTelemetry, Langfuse/Arize Phoenix, OpenLLMetry, NeMo Guardrails, Presidio PII Masking.
6) 	Security & compliance: Zero trust networking, VPC peering, data at rest encryption, IAM policy hardening. 
7) 	Strong AI Engineering expertise: Strong problem-solving capability, ability to evaluate trade-offs between quantization precision vs. latency, and high cross-vendor technical leadership.
8) 	Performance Tuning: Profile GPU kernels with NVIDIA Nsight Systems / DCGM. Fine tune batch sizes, sequence lengths and tensor parallelism for each model. Apply mixed precision (FP16/BF16) and kernel level optimisations (CUDA streams, pinned memory). Use automated load testing (Locust, k6) to benchmark end to end latency under realistic traffic
9) 	Troubleshooting: Set up alerting on latency, GPU memory pressure, OOM, and token rate spikes via Prometheus Alert manager. Maintain runbooks for common failure modes (GPU driver crash, network partition, model version mismatch). 
10) 	AI/ML Stack expertise: Familiarity with AI frameworks and orchestration tools like PyTorch, TensorFlow, LangChain, Hugging Face and Agentic AI platforms. 
11) 	CI/CD & GitOps tooling: Argo CD, Flux, Tekton pipelines
12) 	Edge AI deployment: Ability to ship lightweight inference (onnxruntime, llama.cpp) to edge devices.

Functional Competencies :

Financial Market

Job Purpose :

1) 	The successful candidate would join the IT Department of a critical Financial Market Infrastructure (FMI) organization which serves as a Qualified Central Counterparty (QCCP) for Government Securities, Money Market, FX and Derivatives transactions in the Indian financial markets.
2) 	We are establishing a dedicated Innovation Lab to identify, research and prototype the next generation of capabilities for Indian financial markets. The Lab operates as a fast-moving unit with a distinct mandate: take an idea from problem statement to a validated proof of concept and an approved business case. 
3) 	We are seeking an AI Platform Engineer to design, build, and scale our enterprise AI and LLM infrastructure. Successful candidate will bridge the gap between underlying Sovereign cloud infrastructure and AI application development. Your work ensures that data scientists, machine learning engineers, and product teams can build, test, and run AI workloads reliably, securely, and cost-effectively.

Area of Operations :

Cloud, Platform and AI Engineering

Key Responsibility :

1) 	Infrastructure & Orchestration: Build and manage scalable containerized environments using Kubernetes for heavy AI and LLM workloads
2) 	Enterprise AI Gateway Architecture: Design, build, scale, and maintain CCIL’s centralized AI Gateway providing standardized API contracts (OpenAI/REST-compliant) for downstream applications and orchestration platforms.
3) 	Model Serving & GPU Optimization: Lead deployment of Foundation Models (LLMs, SLMs, Embedding Models, Vision-Language Models) on sovereign GPU clusters. Optimize throughput and latency using vLLM, NVIDIA Triton, TensorRT-LLM, TGI, and advanced quantization (AWQ, GPTQ, FP8).
4) 	Traffic Monitoring & Semantic Routing: Implement high-performance intelligent routing, multi-model failover, load balancing, rate limiting, and semantic caching layers (Redis/Dragonfly) to reduce latency and optimize GPU token economics.
5) 	Enterprise Guardrails & Policy Enforcement: Embed inline security filters, PII redaction, prompt injection mitigations, and compliance guardrails using frameworks such as NeMo Guardrails, Llama Guard, or custom policy engines.
6) 	Telemetry, Observability & Token Auditing: Architect full-lifecycle telemetry tracking token consumption, latency (TTFT, ITL), throughput, hardware GPU metrics (utilization, VRAM, thermal), drift, and cost allocation across business units.
7) 	Cloud Infra AI Platform Interfacing: Collaborate closely with Infra/Cloud vendors on GPU cluster provisioning.
8) 	Governance & Cost Control: Define cost allocation tags, budget alerts and auto scaling policies to keep AI spend within agreed limits.
9) 	Packaging and deployment: Build, Packaging and deployment of AI workloads into UAT and Production environments

Any Other Requirement :

1) 	Should be a good team player.
2) 	Would be required to work with multiple projects / teams concurrently.