Touchcast CogCache FAQ

In this section we'll review the most frequently asked questions from users of Touchcast CogCache.

What is CogCache?

CogCache is an optimized edge caching solution for the Generative Web that provides the most cost-effective, high-performance way to access Azure OpenAI with no capacity limits.

How does CogCache reduce costs?

CogCache can save up to 50% on LLM costs for similar, recurring prompts by serving content from cache, eliminating the need to consume tokens on previously generated content. It also reduces your carbon footprint, making AI operations more sustainable.

Additionally, if you have the plan that provides you access to the Azure OpenAI LLMs then you will get additional cost reductions through the discounts we offer for those LLMs, starting at 5%.

How does CogCache improve performance?

CogCache accelerates response times by up to 100x with its two-tiered system that uses high-speed in-memory hashing and vector search to radically reduce latency and lower token rendering costs.

What observability and control features are available?

CogCache provides a dashboard for teams to monitor the generative flow, set policies, and audit content with confidence. It offers complete transparency and auditability of all generated text, as well as the ability to edit and correct the cache entries.

What flexibility does CogCache provide?

CogCache offers complete flexibility with no monthly commitments required. You can pay only for what you need with monthly subscriptions.

How many flavours of CogCache are?

There are two flavours of CogCache:

CogCache with Caching only

You can purchase a caching only plan and you get all the benefits of CogCache, but without the LLMs. CogCache will require to bring your own Azure OpenAI deployment. In this case we have 3 dedicated plans where you can purchase bundled cache items.

CogCache with Caching and Azure OpenAI LLMs

This is the recommend way to purchase CogCache benefiting the additional cost reductions that goes with using our discounted Azure OpenAI PTU-level deployments. The charging happens for the number of generated tokens with better discounts as the usage within the month grows. In this case we have a single plan with no upfront commitment and you will only pay for what you use at the end of the month.

How quickly can I get started with CogCache?

You can get started with CogCache in minutes by simply pointing your Azure OpenAI calls to your CogCache instance. No code changes or implementation resources are needed.

What kind of throughput can I expect with CogCache?

CogCache ensures throughput is always available at low latency with reserved processing capacity, whether used or not.

How does CogCache's intelligent cache retrieval work?

CogCache uses a two-tiered process with a high-performance in-memory hashing mechanism for fast exact matches, and vector-based semantic search to find semantically similar items, optimizing performance and cache recall.

What insights can I gain from CogCache's observability features?

CogCache allows you to gain real-time insights, track key performance metrics, and view all logged requests for easy debugging with its full-stack LLM observability.

What level of control does the CogCache Dashboard provide?

The CogCache Dashboard interface allows users to explore the cache and enables direct editing of stored responses for precise control.

What reporting and analytics features are available in CogCache?

CogCache provides comprehensive reporting features with insights into cache usage serving as crucial evaluative metrics for data-driven decision-making and system performance enhancements.

How do I get in touch with support questions or inquiries?

You can contact us anytime through our support website. We are always happy to hear from you, and will do our best to help you with any questions or issues you may have.