LLM KEY SERVER: PROVIDING SECURE AND CONVENIENT ACCESS TO INTERNAL LLM APIS

Summary from the TLDR InfoSec newsletter

Mercari's AI Security team developed a key server to address the challenge of managing API key-based access to LLMs. The key server uses Google Workspace and Google Cloud for OIDC authentication to LiteLLM, which provides time-limited API access to various models through a unified API. The team also developed an internal CLI, GitHub Action, and Google Apps Script template to support adoption of the LLM key server.