langchain_community.embeddings.xinference
.XinferenceEmbeddings¶
- class langchain_community.embeddings.xinference.XinferenceEmbeddings(server_url: Optional[str] = None, model_uid: Optional[str] = None)[source]¶
Xinference embedding models.
To use, you should have the xinference library installed:
pip install xinference
Check out: https://github.com/xorbitsai/inference To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers.
Example
To start a local instance of Xinference, run
$ xinference
You can also deploy Xinference in a distributed cluster. Here are the steps:
Starting the supervisor:
$ xinference-supervisor
Starting the worker:
$ xinference-worker
Then, launch a model using command line interface (CLI).
Example:
$ xinference launch -n orca -s 3 -q q4_0
It will return a model UID. Then you can use Xinference Embedding with LangChain.
Example:
from langchain_community.embeddings import XinferenceEmbeddings xinference = XinferenceEmbeddings( server_url="http://0.0.0.0:9997", model_uid = {model_uid} # replace model_uid with the model UID return from launching the model )
Attributes
client
server_url
URL of the xinference server
model_uid
UID of the launched model
Methods
__init__
([server_url, model_uid])aembed_documents
(texts)Asynchronous Embed search docs.
aembed_query
(text)Asynchronous Embed query text.
embed_documents
(texts)Embed a list of documents using Xinference.
embed_query
(text)Embed a query of documents using Xinference.
- async aembed_documents(texts: List[str]) List[List[float]] ¶
Asynchronous Embed search docs.
- async aembed_query(text: str) List[float] ¶
Asynchronous Embed query text.