agentgateway

Overview

agentgateway is an open source HTTP and gRPC gateway that handles traditional application traffic and AI-native protocols in one data plane. It provides routing, security, observability, governance, and cost controls for services, LLM provider traffic, MCP tools, and agent-to-agent communication, reducing the need to stitch together separate gateways.

Key Features

• One data plane for HTTP, gRPC, and TCP services, LLM providers, self-hosted inference, MCP, and A2A traffic.
• LLM gateway with OpenAI-compatible API support, provider routing and failover, token budgets, semantic caching, and prompt redaction.
• MCP gateway capabilities including server discovery, tool-call signing and scoping, versioning, and audit trails.
• A2A routing across frameworks such as LangChain, CrewAI, and ADK with identity and tracing on every hop.
• Security controls including JWT/OIDC, API keys, external authorization, request authorization, TLS/mTLS, CORS/CSRF, and rate limiting.
• Observability and cost management with OpenTelemetry, Prometheus, Jaeger, token metrics, realized USD cost, virtual keys, budgets, and an admin dashboard.

Use Cases

• Consolidate north-south and east-west service traffic with AI and agent traffic behind a single gateway.
• Govern LLM provider access with budgets, virtual keys, cost tracking, and prompt guards.
• Manage MCP server discovery, tool permissions, and audit trails for agent tool use.
• Route agent-to-agent invocations between frameworks such as LangChain, CrewAI, ADK, and custom runtimes.

Getting Started

• Install the agentgateway binary using the published install script.
• Follow the standalone or Kubernetes quick start, which the site describes as taking less than 5 minutes.
• Use the MCP connectivity guide to connect agentgateway to MCP servers with discovery and auth.
• Try the LLM playground from the admin UI to test a model route before shipping it.

Deployment & Requirements

• Deployment options include a binary, Docker, or Kubernetes.
• Standalone and Kubernetes documentation and quick starts are provided.

Before You Adopt

• License: Apache-2.0. Review its terms before using, modifying, or distributing the project.
• The LLM gateway is designed to sit in front of external model providers and self-hosted models, so deployments depend on the configured upstream providers and credentials.
• Prompt guards can perform pattern and model-based checks and redact sensitive content before it leaves the environment.
• Rate limiting is available to protect backends from abuse and runaway agent loops.