langchain_community.cache
.AstraDBSemanticCache¶
- class langchain_community.cache.AstraDBSemanticCache(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)[source]¶
Cache that uses Astra DB as a vector-store backend for semantic (i.e. similarity-based) lookup.
It uses a single (vector) collection and can store cached values from several LLMs, so the LLMâs âllm_stringâ is stored in the document metadata.
You can choose the preferred similarity (or use the API default) â remember the threshold might require metric-dependend tuning.
Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint,
such as âhttps://<DB-ID>-us-east1.apps.astra.datastax.comâ.
- Parameters
astra_db_client (Optional[Any]) â alternative to token+api_endpoint, you can pass an already-created âastrapy.db.AstraDBâ instance.
namespace (Optional[str]) â namespace (aka keyspace) where the collection is created. Defaults to the databaseâs âdefault namespaceâ.
embedding (Embedding) â Embedding provider for semantic encoding and search.
metric â the function to use for evaluating similarity of text embeddings. Defaults to âcosineâ (alternatives: âeuclideanâ, âdot_productâ)
similarity_threshold (float, optional) â the minimum similarity for accepting a (semantic-search) match.
The default score threshold is tuned to the default metric. Tune it carefully yourself if switching to another distance metric.
Methods
__init__
(*[, collection_name, token, ...])Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint, such as "https://<DB-ID>-us-east1.apps.astra.datastax.com". :type api_endpoint: Optional[str] :param astra_db_client: alternative to token+api_endpoint, you can pass an already-created 'astrapy.db.AstraDB' instance. :type astra_db_client: Optional[Any] :param namespace: namespace (aka keyspace) where the collection is created. Defaults to the database's "default namespace". :type namespace: Optional[str] :param embedding: Embedding provider for semantic encoding and search. :type embedding: Embedding :param metric: the function to use for evaluating similarity of text embeddings. Defaults to 'cosine' (alternatives: 'euclidean', 'dot_product') :param similarity_threshold: the minimum similarity for accepting a (semantic-search) match. :type similarity_threshold: float, optional.
clear
(**kwargs)Clear the whole semantic cache.
delete_by_document_id
(document_id)Given this is a "similarity search" cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID.
lookup
(prompt, llm_string)Look up based on prompt and llm_string.
lookup_with_id
(prompt, llm_string)Look up based on prompt and llm_string.
lookup_with_id_through_llm
(prompt, llm[, stop])update
(prompt, llm_string, return_val)Update cache based on prompt and llm_string.
- __init__(*, collection_name: str = 'langchain_astradb_semantic_cache', token: Optional[str] = None, api_endpoint: Optional[str] = None, astra_db_client: Optional[Any] = None, namespace: Optional[str] = None, embedding: Embeddings, metric: Optional[str] = None, similarity_threshold: float = 0.85)[source]¶
Initialize the cache with all relevant parameters. :param collection_name: name of the Astra DB collection to create/use. :type collection_name: str :param token: API token for Astra DB usage. :type token: Optional[str] :param api_endpoint: full URL to the API endpoint,
such as âhttps://<DB-ID>-us-east1.apps.astra.datastax.comâ.
- Parameters
astra_db_client (Optional[Any]) â alternative to token+api_endpoint, you can pass an already-created âastrapy.db.AstraDBâ instance.
namespace (Optional[str]) â namespace (aka keyspace) where the collection is created. Defaults to the databaseâs âdefault namespaceâ.
embedding (Embedding) â Embedding provider for semantic encoding and search.
metric â the function to use for evaluating similarity of text embeddings. Defaults to âcosineâ (alternatives: âeuclideanâ, âdot_productâ)
similarity_threshold (float, optional) â the minimum similarity for accepting a (semantic-search) match.
The default score threshold is tuned to the default metric. Tune it carefully yourself if switching to another distance metric.
- delete_by_document_id(document_id: str) None [source]¶
Given this is a âsimilarity searchâ cache, an invalidation pattern that makes sense is first a lookup to get an ID, and then deleting with that ID. This is for the second step.
- lookup(prompt: str, llm_string: str) Optional[Sequence[Generation]] [source]¶
Look up based on prompt and llm_string.
- lookup_with_id(prompt: str, llm_string: str) Optional[Tuple[str, Sequence[Generation]]] [source]¶
Look up based on prompt and llm_string. If there are hits, return (document_id, cached_entry) for the top hit
- lookup_with_id_through_llm(prompt: str, llm: LLM, stop: Optional[List[str]] = None) Optional[Tuple[str, Sequence[Generation]]] [source]¶
- update(prompt: str, llm_string: str, return_val: Sequence[Generation]) None [source]¶
Update cache based on prompt and llm_string.