Tetrate and Bloomberg Unveil Open Source Envoy AI Gateway

Tetrate and Bloomberg have released the first stable version of the open source Envoy AI Gateway, built on CNCF's Envoy Gateway project, announced in a press release. This initiative aims to expand Envoy's capabilities and accelerate generative AI application development.

The Envoy AI Gateway, now available for download on GitHub, provides a unified API interface for developers integrating applications with generative AI services. It supports integrations with AWS Bedrock and OpenAI, offering features like upstream authorization and usage rate limiting based on word tokens.

Bloomberg is utilizing the Envoy AI Gateway to scale its generative AI application development, managing GenAI services through a consistent API. This approach simplifies development and enhances the company's ability to deliver innovative AI services.

Future enhancements for the Envoy AI Gateway include Google Gemini 2.0 integration, provider and model fallback logic, prompt templating, and semantic caching. These features aim to ensure service continuity and reduce costs associated with LLM interactions.