In this section we'll review the most frequently asked questions from users of Touchcast CogCache.
Don’t see your question below? Leave us a comment; we believe in real, human connection, and one of our support team agents will respond as soon as possible.
What is CogCache?
CogCache is an optimized edge caching solution for the Generative Web that provides the most cost-effective, high-performance way to access Azure OpenAI with no capacity limits.
How does CogCache reduce costs?
CogCache can save up to 50% on LLM costs for similar, recurring prompts by serving content from cache, eliminating the need to consume tokens on previously generated content. It also reduces your carbon footprint, making AI operations more sustainable.
How does CogCache improve performance?
CogCache accelerates response times by up to 100x with its two-tiered system that uses high-speed in-memory hashing and vector search to radically reduce latency and lower token rendering costs.
What alignment and safety features does CogCache offer?
CogCache uses Dynamic Constitutional AI to analyze and score cached content asynchronously, ensuring alignment and grounding of responses. It also has a self-healing mechanism to identify and mitigate misaligned responses automatically or with human approval.
What observability and control features are available?
CogCache provides a dashboard for teams to monitor the generative flow, set policies, and audit content with confidence. It offers complete transparency and auditability of all generated text, as well as the ability to edit, align, and correct cache entries.
How does CogCache handle temporally relevant content?
CogCache has a Temporal Relevance Tracking feature that can discern if a cached result might no longer be applicable based on its content and automatically refresh items as needed.
What flexibility does CogCache provide?
CogCache offers complete flexibility with no monthly commitments required. You can pay only for what you need with monthly and annual subscriptions.
How quickly can I get started with CogCache?
You can get started with CogCache in minutes by simply pointing your Azure OpenAI calls to your CogCache instance. No code changes or implementation resources are needed.
What kind of throughput can I expect with CogCache?
CogCache ensures throughput is always available at low latency with reserved processing capacity, whether used or not.
How does CogCache's intelligent cache retrieval work?
CogCache uses a two-tiered process with a high-performance in-memory hashing mechanism for fast exact matches, and vector-based semantic search to find semantically similar items, optimizing performance and cache recall.
What insights can I gain from CogCache's observability features?
CogCache allows you to gain real-time insights, track key performance metrics, and view all logged requests for easy debugging with its full-stack LLM observability.
How does CogCache's asynchronous alignment scoring work?
CogCache operates asynchronously to provide continuous alignment assessment of cached content using Dynamic Constitutional AI principles, ensuring adherence to safety, relevance, and alignment standards without sacrificing performance.
What level of control does the CogCache Dashboard provide?
The CogCache Dashboard interface allows users to explore the cache, offers explainability of the AI's decision-making and alignment scores, and enables direct AI-assisted editing of stored responses for precise control.
How does CogCache handle AI governance and compliance?
CogCache's Dynamic Constitutional AI construct allows the cache to evolve and adapt to changing norms, standards, and governance requirements, with periodic updates based on Constitutional Amendments for various use cases and verticals.
What reporting and analytics features are available in CogCache?
The CogCache Copilot provides comprehensive reporting features with insights into cache usage and response quality, serving as crucial evaluative metrics for data-driven decision-making and system performance enhancements.
How do I get in touch with support questions or inquiries?
You can contact us anytime through our support website. We are always happy to hear from you, and will do our best to help you with any questions or issues you may have.