langchain_community.document_loaders.rss.RSSFeedLoader¶

class langchain_community.document_loaders.rss.RSSFeedLoader(urls: Optional[Sequence[str]] = None, opml: Optional[str] = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any)[source]¶

Load news articles from RSS feeds using Unstructured.

Parameters

  • urls (Optional[Sequence[str]]) – URLs for RSS feeds to load. Each articles in the feed is loaded into its own document.

  • opml (Optional[str]) – OPML file to load feed urls from. Only one of urls or opml should be provided. The value

  • string (can be a URL) –

  • string. (or OPML markup contents as byte or) –

  • continue_on_failure (bool) – If True, continue loading documents even if loading fails for a particular URL.

  • show_progress_bar (bool) – If True, use tqdm to show a loading progress bar. Requires tqdm to be installed, pip install tqdm.

  • **newsloader_kwargs (Any) – Any additional named arguments to pass to NewsURLLoader.

Example

from langchain_community.document_loaders import RSSFeedLoader

loader = RSSFeedLoader(
    urls=["<url-1>", "<url-2>"],
)
docs = loader.load()

The loader uses feedparser to parse RSS feeds. The feedparser library is not installed by default so you should install it if using this loader: https://pythonhosted.org/feedparser/

If you use OPML, you should also install listparser: https://pythonhosted.org/listparser/

Finally, newspaper is used to process each article: https://newspaper.readthedocs.io/en/latest/

Initialize with urls or OPML.

Methods

__init__([urls, opml, continue_on_failure, ...])

Initialize with urls or OPML.

alazy_load()

A lazy loader for Documents.

lazy_load()

A lazy loader for Documents.

load()

Load data into Document objects.

load_and_split([text_splitter])

Load Documents and split into chunks.
__init__(urls: Optional[Sequence[str]] = None, opml: Optional[str] = None, continue_on_failure: bool = True, show_progress_bar: bool = False, **newsloader_kwargs: Any) → None[source]¶

Initialize with urls or OPML.

Parameters

  • urls (Optional[Sequence[str]]) –

  • opml (Optional[str]) –

  • continue_on_failure (bool) –

  • show_progress_bar (bool) –

  • newsloader_kwargs (Any) –

Return type

None

async alazy_load() → AsyncIterator[Document]¶

A lazy loader for Documents.

Return type

AsyncIterator[Document]

lazy_load() → Iterator[Document][source]¶

A lazy loader for Documents.

Return type

Iterator[Document]

load() → List[Document][source]¶

Load data into Document objects.

Return type

List[Document]

load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns

List of Documents.

Return type

List[Document]

Examples using RSSFeedLoader¶