Proxy API

Proxy API Overview

The Proxy API is the API that allows proxying of the Azure OpenAI deployment.

To get started you only have to replace the Azure OpenAI LLM call with a call to the proxy. If no answer found in the cache, the proxy will then forward the request to the LLM and return the response to the user.

Request headers

Cache-Control

This header affects the cache behaviour as follows:

• no-store - disables storing the answer from the LLM in the cache

• no-cache - disables retrieval of answer from cache, goes directly to the LLM

• only-if-cached - retrieves answer only from the cache and if not found it returns 504 error code

CogCache-Auth

This header is used to authenticate and authorise the CogCache Proxy API. It helps CogCache identify the organization and application.

The format is like this: Bearer COGCACHE_APIKEY

CogCache-OpenAI-Api-Base

This is the URL to your Azure OpenAI deployment, in this format https://YOUR_RESOURCE_NAME.openai.azure.com

Response headers

X-Cache

This header tells if the response was provided from the cache or not. However, the header will not be available if Cache-Control has a value of no-store or no-cache.

Possibles values:

• hit - the answer was retrieved from cache

• miss - the answer was retrieved directly from the LLM

CogCache-Cache-Entry-ID

This response header holds the cache entry unique ID when the response is retrieved from cache. When the response is retrieved from the LLM this header will not be available.

Error messages