Back GenAI Engineer IT Infrastructure & Operations

​Contribute to the design, development, and operations of the organization's AI platform, spanning LLM infrastructure, agent systems, and AI Platforms like Glean (enterprise search platform) or Gemini Enterprise Apps or Claude Cowork

Support the Glean or any other AI platform by driving sharing, agent development, Glean enablement, and collaboration with the vendors on new feature implementation

Help build and maintain the LLM gateway (LiteLLM-based), including multi-provider model routing, fallback configuration, caching, cost tracking, FinOps, and guardrail integration (e.g., Amazon Bedrock guardrails)

Develop and maintain LLM observability capabilities — prompt/response logging, token and cost attribution, latency tracking, failure mode clustering, hallucination detection, and input drift monitoring

Build, test, and iterate on AI agents and agentic workflows, including multi-step tool use, orchestration patterns, error handling, and human-in-the-loop mechanisms

Integrate and manage MCP (Model Context Protocol) servers to connect agents and LLM applications with external tools and data sources such as Slack, Jira, databases, and internal APIs

Design and execute evaluation frameworks for LLM applications and agents — building golden datasets, implementing LLM-as-judge patterns, running regression tests on prompt and model changes, and reporting on quality metrics

Support VectorDB infrastructure including ingestion pipelines, chunking strategies, retrieval quality measurement, and integration with the broader AI platform

Maintain infrastructure-as-code (Terraform), CI/CD pipelines (GitHub Actions), and cloud resources (AWS) that underpin the AI platform

• Collaborate with Data Science, Product, and Engineering teams to understand use cases, resolve platform issues, and continuously improve the developer experience for AI application builders across the organization