BT Group’s Digital Unit has introduced an internal platform named GenAI Gateway, designed to leverage large language models (LLMs) from various providers such as Anthropic, Meta, Claude, Cohere, and Amazon. This platform was developed in collaboration with AWS using Amazon Bedrock, Amazon SageMaker, and AWS Professional Services. The GenAI Gateway offers secure access to natural-language processing and LLMs, which BT Group plans to integrate into its business operations.

The use of LLMs for test and development is suitable but poses challenges for large-scale applications due to cost control, security, and privacy concerns. Additionally, monitoring LLM performance for errors and model decay is essential. The GenAI Gateway addresses these issues by preventing dependency on a single LLM provider and enabling engineers to select the appropriate model based on specific needs and budget constraints.

A unified platform like GenAI Gateway minimizes resource duplication as BT Group expands its generative AI usage. Centralized management of APIs, security configurations, and infrastructure reduces errors and maintenance costs associated with separate LLMs for different use cases.

Deployed on AWS, the GenAI Gateway utilizes Amazon Bedrock to offer high-performing foundation models from AI companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API. It also employs Amazon SageMaker for low-cost machine learning tools applicable to various use cases. The platform supports features like prompt security, chat history management, FinOps billing per use case, enterprise search capabilities, and integration with multiple corporate data sources. Privacy controls include isolated tenants for each use case and filters for personal identifiable information while ensuring data remains within the UK.

The GenAI Gateway is live with initial beta use cases. One trial at Openreach involves summarizing engineering notes on Ethernet and full fiber jobs to streamline processes and enhance productivity. Another application supports contract analysis for BT Group's business units including legal and procurement teams.

Fabio Cerone from AWS commented: “The BT Group GenAI Gateway is showcasing how enterprises can effectively deploy generative AI at scale and speed. It’s been a brilliant opportunity to collaborate... providing a way to accelerate deployment of generative AI use cases into production with embedded security and compliance.”

Deepika Adusumilli of BT Group’s Digital Unit stated: “GenAI Gateway allows us to tap into this powerful new set of technologies at scale... delivering the ambition we have for AI to unlock the human potential within BT Group.”

BT Group provides fixed and mobile telecommunications services in the UK along with managed telecommunications services internationally through three customer-facing units: Consumer (for individuals), Business (for companies), and Openreach (wholesale fixed access infrastructure).