langchain_community.cache.AstraDBSemanticCache

class langchain_community.cache.AstraDBSemanticCache(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)

Cache that uses Astra DB as a vector-store backend for semantic (i.e. similarity-based) lookup.

It uses a single (vector) collection and can store cached values from several LLMs, so the LLM’s ‘llm_string’ is stored in the document metadata.

You can choose the preferred similarity (or use the API default) – remember the threshold might require metric-dependend tuning.

Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint,

such as “https://<DB-ID>-us-east1.apps.astra.datastax.com”.

Parameters

• astra_db_client (Optional[Any]) – alternative to token+api_endpoint, you can pass an already-created ‘astrapy.db.AstraDB’ instance.

• namespace (Optional[str]) – namespace (aka keyspace) where the collection is created. Defaults to the database’s “default namespace”.

• embedding (Embedding) – Embedding provider for semantic encoding and search.

• metric – the function to use for evaluating similarity of text embeddings. Defaults to ‘cosine’ (alternatives: ‘euclidean’, ‘dot_product’)

• similarity_threshold (float, optional) – the minimum similarity for accepting a (semantic-search) match.

The default score threshold is tuned to the default metric. Tune it carefully yourself if switching to another distance metric.

Methods

__init__(*[, collection_name, token, ...])	Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint, such as "https://<DB-ID>-us-east1.apps.astra.datastax.com". :type api_endpoint: Optional[str] :param astra_db_client: alternative to token+api_endpoint, you can pass an already-created 'astrapy.db.AstraDB' instance. :type astra_db_client: Optional[Any] :param namespace: namespace (aka keyspace) where the collection is created. Defaults to the database's "default namespace". :type namespace: Optional[str] :param embedding: Embedding provider for semantic encoding and search. :type embedding: Embedding :param metric: the function to use for evaluating similarity of text embeddings. Defaults to 'cosine' (alternatives: 'euclidean', 'dot_product') :param similarity_threshold: the minimum similarity for accepting a (semantic-search) match. :type similarity_threshold: float, optional.
clear(**kwargs)	Clear the whole semantic cache.
delete_by_document_id(document_id)	Given this is a "similarity search" cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID.
lookup(prompt, llm_string)	Look up based on prompt and llm_string.
lookup_with_id(prompt, llm_string)	Look up based on prompt and llm_string.
lookup_with_id_through_llm(prompt, llm[, stop])
update(prompt, llm_string, return_val)	Update cache based on prompt and llm_string.

__init__(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)

Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint,

such as “https://<DB-ID>-us-east1.apps.astra.datastax.com”.

Parameters

• astra_db_client (Optional[Any]) – alternative to token+api_endpoint, you can pass an already-created ‘astrapy.db.AstraDB’ instance.

• namespace (Optional[str]) – namespace (aka keyspace) where the collection is created. Defaults to the database’s “default namespace”.

• embedding (Embedding) – Embedding provider for semantic encoding and search.

• metric – the function to use for evaluating similarity of text embeddings. Defaults to ‘cosine’ (alternatives: ‘euclidean’, ‘dot_product’)

• similarity_threshold (float, optional) – the minimum similarity for accepting a (semantic-search) match.

The default score threshold is tuned to the default metric. Tune it carefully yourself if switching to another distance metric.

clear(**kwargs: Any) → None

Clear the whole semantic cache.

delete_by_document_id(document_id: str) → None

Given this is a “similarity search” cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID. This is for the second step.

lookup(prompt: str, llm_string: str) → Optional[Sequence[Generation]]

Look up based on prompt and llm_string.

lookup_with_id(prompt: str, llm_string: str) → Optional[Tuple[str, Sequence[Generation]]]

Look up based on prompt and llm_string. If there are hits, return (document_id, cached_entry) for the top hit

lookup_with_id_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) → Optional[Tuple[str, Sequence[Generation]]]

update(prompt: str, llm_string: str, return_val: Sequence[Generation]) → None

Update cache based on prompt and llm_string.