langchain_community.document_loaders.base
.BaseLoader¶
- class langchain_community.document_loaders.base.BaseLoader[source]¶
Interface for Document Loader.
Implementations should implement the lazy-loading method using generators to avoid loading all Documents into memory at once.
The load method will remain as is for backwards compatibility, but its implementation should be just list(self.lazy_load()).
Methods
__init__
()A lazy loader for Documents.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
- __init__()¶
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document] [source]¶
Load Documents and split into chunks. Chunks are returned as Documents.
- Parameters
text_splitter – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.