langchain.evaluation.loading.load_evaluators

langchain.evaluation.loading.load_evaluators(evaluators: Sequence[EvaluatorType], *, llm: Optional[BaseLanguageModel] = None, config: Optional[dict] = None, **kwargs: Any) → List[Union[Chain, StringEvaluator]][source]

Load evaluators specified by a list of evaluator types.

Parameters

  • evaluators (Sequence[EvaluatorType]) – The list of evaluator types to load.

  • llm (BaseLanguageModel, optional) – The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used.

  • config (dict, optional) – A dictionary mapping evaluator types to additional keyword arguments, by default None

  • **kwargs (Any) – Additional keyword arguments to pass to all evaluators.

Returns

The loaded evaluators.

Return type

List[Chain]

Examples

>>> from langchain.evaluation import load_evaluators, EvaluatorType
>>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA]
>>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness")