New features

Similar prompts

A similar prompt represents a rephrase of the main prompt for a cache item while maintaining the same meaning with the purpose of increasing the chances for that cache item to be returned.

When a request is made to the Proxy API the system will do exact match and similarity match of both the main prompt and similar prompts thus making it more likely for the cache items with similar prompts to have higher hit rates than the ones without any.

Similar prompts can be added manually via the CogCache CMS, or automatically by the system, when a specific cache item was returned via similarity match.

Currently the system supports an unlimited amounts of similar prompts.

Below it’s a screenshot of the manage single cache item screen that shows the area where you can manage the similar prompts.

Open

The screenshot below shows list of all cache items and how is easy is to identify when a cache item that has similar prompts. There is no need to go into manage mode to know that. Also important to note that when searching on this screen for cache item, you can also search for similar prompts and the corresponding cache item will be returned.

Open

Export cache items

We have introduced on the cache items list page the ability to export cache items to a CSV file.

You can export all the cache items, or you can limit the export to specific applications.

Good to know that the exported CSV file can be used to import cache items via the import CSV functionality.

Bulk delete cache items

We have made it easier to delete cache items by introducing the ability to delete them in bulk.

To delete multiple cache items at a time, click the checkbox button for each corresponding cache item that you want to delete. Then click the button, top right.

Similarity threshold

We have introduced the ability to control the similarity threshold at the application level. Now when you create or edit an application you can change this threshold with a range of values between 0 and 99%.

When there is a Proxy API request and we don’t have an exact match we employ similarity search. The sensitivity of the search is controlled by this similarity threshold setting. Higher values mean more precise matches with less risk of returning inadequate responses.

By default we set a value of 82%, which is subject to change.

Unique URLs for paginated cache items and filters

For the cache items page we have introduced unique URLs for each page and also unique URLs for when filters as applied.

The benefit is that you can now share a specific list of cache items with someone else by simply copying the URL from the browser. It also helps with maintaining the state of the list when going into manage mode for a cache item and return back to the list.

In the screenshot below you can see such a unique URL for when user is searching cache items that contain the word “ai“, are part of a specific application and also navigated to the second page.

Improvements

CSV import

Now when we import a CSV file we ask the user to chose which are the columns from the CSV that contain the prompt and answer. By taking this approach we provide flexibility needed to import a wide variety of CSV formats.

Exact match and similarity search

We’ve slightly improved the exact match and similarity search to return better results by not taking into account punctuation marks at the end of the prompt such as: ?, !, ., ,, ; and also by cleaning up some special unicode characters from the prompt.