langchain_community.document_loaders.lakefs
.LakeFSLoader¶
- class langchain_community.document_loaders.lakefs.LakeFSLoader(lakefs_access_key: str, lakefs_secret_key: str, lakefs_endpoint: str, repo: Optional[str] = None, ref: Optional[str] = 'main', path: Optional[str] = '')[source]¶
Load from lakeFS.
- Parameters
lakefs_access_key (str) – [required] lakeFS server’s access key
lakefs_secret_key (str) – [required] lakeFS server’s secret key
lakefs_endpoint (str) – [required] lakeFS server’s endpoint address, ex: https://example.my-lakefs.com
repo (str) – [optional, default = ‘’] target repository
ref (str) – [optional, default = ‘main’] target ref (branch name, tag, or commit ID)
path (str) – [optional, default = ‘’] target path
Attributes
repo
ref
path
Methods
__init__
(lakefs_access_key, ...[, repo, ...])- param lakefs_access_key
[required] lakeFS server's access key
A lazy loader for Documents.
A lazy loader for Documents.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
set_path
(path)set_ref
(ref)set_repo
(repo)- __init__(lakefs_access_key: str, lakefs_secret_key: str, lakefs_endpoint: str, repo: Optional[str] = None, ref: Optional[str] = 'main', path: Optional[str] = '')[source]¶
- Parameters
lakefs_access_key (str) – [required] lakeFS server’s access key
lakefs_secret_key (str) – [required] lakeFS server’s secret key
lakefs_endpoint (str) – [required] lakeFS server’s endpoint address, ex: https://example.my-lakefs.com
repo (Optional[str]) – [optional, default = ‘’] target repository
ref (Optional[str]) – [optional, default = ‘main’] target ref (branch name, tag, or commit ID)
path (Optional[str]) – [optional, default = ‘’] target path
- async alazy_load() AsyncIterator[Document] ¶
A lazy loader for Documents.
- Return type
AsyncIterator[Document]
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document] ¶
Load Documents and split into chunks. Chunks are returned as Documents.
Do not override this method. It should be considered to be deprecated!
- Parameters
text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.
- Return type
List[Document]