langchain_community.document_loaders.rss
.RSSFeedLoader¶
- class langchain_community.document_loaders.rss.RSSFeedLoader(urls: Optional[Sequence[str]] = None, opml: Optional[str] = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any)[source]¶
Load news articles from RSS feeds using Unstructured.
- Parameters
urls – URLs for RSS feeds to load. Each articles in the feed is loaded into its own document.
opml – OPML file to load feed urls from. Only one of urls or opml should be provided. The value
string (can be a URL) –
string. (or OPML markup contents as byte or) –
continue_on_failure – If True, continue loading documents even if loading fails for a particular URL.
show_progress_bar – If True, use tqdm to show a loading progress bar. Requires tqdm to be installed,
pip install tqdm
.**newsloader_kwargs – Any additional named arguments to pass to NewsURLLoader.
Example
from langchain_community.document_loaders import RSSFeedLoader loader = RSSFeedLoader( urls=["<url-1>", "<url-2>"], ) docs = loader.load()
The loader uses feedparser to parse RSS feeds. The feedparser library is not installed by default so you should install it if using this loader: https://pythonhosted.org/feedparser/
If you use OPML, you should also install listparser: https://pythonhosted.org/listparser/
Finally, newspaper is used to process each article: https://newspaper.readthedocs.io/en/latest/
Initialize with urls or OPML.
Methods
__init__
([urls, opml, continue_on_failure, ...])Initialize with urls or OPML.
A lazy loader for Documents.
load
()Load data into Document objects.
load_and_split
([text_splitter])Load Documents and split into chunks.
- __init__(urls: Optional[Sequence[str]] = None, opml: Optional[str] = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any) None [source]¶
Initialize with urls or OPML.
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document] ¶
Load Documents and split into chunks. Chunks are returned as Documents.
- Parameters
text_splitter – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.