langchain_community.document_loaders.url_selenium.SeleniumURLLoader

class langchain_community.document_loaders.url_selenium.SeleniumURLLoader(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])

Load HTML pages with Selenium and parse with Unstructured.

This is useful for loading pages that require javascript to render.

urls

List of URLs to load.

Type

List[str]

continue_on_failure

If True, continue loading other URLs on failure.

Type

bool

browser

The browser to use, either ‘chrome’ or ‘firefox’.

Type

str

binary_location

The location of the browser binary.

Type

Optional[str]

executable_path

The path to the browser executable.

Type

Optional[str]

headless

If True, the browser will run in headless mode.

Type

bool

arguments [List[str]]

List of arguments to pass to the browser.

Load a list of URLs using Selenium and unstructured.

Methods

__init__(urls[, continue_on_failure, ...])	Load a list of URLs using Selenium and unstructured.
lazy_load()	A lazy loader for Documents.
load()	Load the specified URLs using Selenium and create Document instances.
load_and_split([text_splitter])	Load Documents and split into chunks.

__init__(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])

Load a list of URLs using Selenium and unstructured.

lazy_load() → Iterator[Document]

A lazy loader for Documents.

load() → List[Document]

Load the specified URLs using Selenium and create Document instances.

Returns

A list of Document instances with loaded content.

Return type

List[Document]

load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]

Load Documents and split into chunks. Chunks are returned as Documents.

Parameters

text_splitter – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns

List of Documents.

Examples using SeleniumURLLoader

• URL