langchain_community.document_loaders.url_selenium.SeleniumURLLoader¶
- class langchain_community.document_loaders.url_selenium.SeleniumURLLoader(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])[source]¶
Load HTML pages with Selenium and parse with Unstructured.
This is useful for loading pages that require javascript to render.
- urls¶
List of URLs to load.
- Type
List[str]
- continue_on_failure¶
If True, continue loading other URLs on failure.
- Type
bool
- browser¶
The browser to use, either ‘chrome’ or ‘firefox’.
- Type
str
- binary_location¶
The location of the browser binary.
- Type
Optional[str]
- executable_path¶
The path to the browser executable.
- Type
Optional[str]
- headless¶
If True, the browser will run in headless mode.
- Type
bool
- arguments [List[str]]
List of arguments to pass to the browser.
Load a list of URLs using Selenium and unstructured.
Methods
__init__(urls[, continue_on_failure, ...])Load a list of URLs using Selenium and unstructured.
A lazy loader for Documents.
load()Load the specified URLs using Selenium and create Document instances.
load_and_split([text_splitter])Load Documents and split into chunks.
- __init__(urls: List[str], continue_on_failure: bool = True, browser: Literal['chrome', 'firefox'] = 'chrome', binary_location: Optional[str] = None, executable_path: Optional[str] = None, headless: bool = True, arguments: List[str] = [])[source]¶
Load a list of URLs using Selenium and unstructured.
- load() List[Document][source]¶
Load the specified URLs using Selenium and create Document instances.
- Returns
A list of Document instances with loaded content.
- Return type
List[Document]
- load_and_split(text_splitter: Optional[TextSplitter] = None) List[Document]¶
Load Documents and split into chunks. Chunks are returned as Documents.
- Parameters
text_splitter – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.
- Returns
List of Documents.