About the Event:
You will learn how to implement token tracking, monitoring, and attribution across your entire AI estate — whether consumption runs through Amazon Bedrock, self-hosted models on your own compute, or third-party APIs such as Anthropic or OpenAI. You will leave with a production-ready implementation pattern that gives you a single pane of glass across every consumption environment.
AWS AI Infrastructure — the foundation layer and native tools for agent cost optimization
AWS AI infrastructure is the layer every AI service runs on: Amazon Bedrock for managed model access, Amazon EKS for self-hosted model serving, and the underlying GPU and CPU compute. Where does each one make economic sense, and how to implement and instrument the choices. Introduction to AWS native tools for token cost optimization available in AWS AgentCore and Bedrock.
Kong AI Gateway — routing, budgets, and guardrails
A single governed entry point in front of every model provider: semantic and cost-aware routing, token-based rate limiting, per-team spend quotas, semantic caching, and prompt guarding, across managed APIs and self-hosted vLLM endpoints alike.
Arize AI / Phoenix — observability and evaluation
Per-request tracing of tokens, cost, and latency alongside output quality: dataset evaluation, drift detection, and quality alerting, so a cost decision is never made blind to its effect on the answer.
If you are asking any of these questions, this workshop is for you
✓ How do I trace token consumption per request across Bedrock, OpenAI, and my self-hosted vLLM endpoints?
✓ How do I set per-agent token budgets and circuit breakers before agentic loops run away?
✓ How do I route requests between managed APIs and self-hosted models based on cost, latency, or quality?
✓ How do I attribute AI costs back to specific teams, services, and use cases in my platform?
What You Will Learn
▶ Consumption and cost attribution — trace every token back to a team, service, and use case across Bedrock, self-hosted models, and third-party APIs
▶ Routing — direct requests across multiple models and providers based on cost, latency, or availability
▶ Governance and monitoring — enforce per-team token budgets, evaluate model datasets, and set up drift detection and quality alerts
What You Will Build
A token tracking and observability stack using Kong AI Gateway and the Arize AI observability stack on Amazon EKS. By the end of the day you will have a working pipeline capturing per-request token counts, cost, latency, model quality, and team attribution — across Amazon Bedrock, foundation model providers accessed directly, and self-hosted vLLM endpoints.
Who Should Attend
■ AI/ML engineers deploying LLMs, agents, and RAG pipelines to production
■ Platform engineers & AI Ops building and operating multi-model serving infrastructure
■ DevOps & SRE teams responsible for AI workload reliability, cost, and observability
■ Backend engineers integrating LLM calls (Bedrock, OpenAI, self-hosted) into applications
■ Engineering leads making build-vs-buy decisions on model serving, routing, and observability
Please bring your own laptop. A lab environment is provided — no local setup is required. Working familiarity with Kubernetes and with calling an LLM API is assumed.
Full Agenda:
Start End Duration Session
9.00 9.30 0.30 Registration & Networking
9.30 10.15 0.45 Token Economics Framework
10.15 10.45 0.30 AWS Services for Token Economics - AgentCore Gateway
10.45 11.00 0.15 B R E A K
11.00 12.00 1.00 Kong AI Gateway — Overview
12.00 13.00 1.00 Arize AI / Phoenix — Overview
13.00 14.00 1.00 L U N C H
14.00 14.30 0.30 Live Demo : Integrated / Unified Token Economics Solution
14.30 15.30 1.00 Hands-On Lab : Integration Workshop — Kong + Arize + AWS End-to-End
15.30 15.45 0.15 B R E A K
15.45 16.45 1.00 Continuation - Hands-On Lab : Integration Workshop — Kong + Arize + AWS End-to-End
16.45 16.55 0.10 Recap and Quiz
16.55 17.00 0.05 W R A P - U P