langchain.smith.evaluation.string_run_evaluator.ChainStringRunMapper

class langchain.smith.evaluation.string_run_evaluator.ChainStringRunMapper[source]

Bases: StringRunMapper

Extract items to evaluate from the run object from a chain.

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param input_key: Optional[str] = None

The key from the model Run’s inputs to use as the eval input. If not provided, will use the only input key or raise an error if there are multiple.

param prediction_key: Optional[str] = None

The key from the model Run’s outputs to use as the eval prediction. If not provided, will use the only output key or raise an error if there are multiple.

__call__(run: Run) → Dict[str, str]

Maps the Run to a dictionary.

classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model

Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model

Duplicate a model, optionally choose which fields to include, exclude and change.

Parameters

  • include – fields to include in new model

  • exclude – fields to exclude from new model, as with values this takes precedence over include

  • update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

  • deep – set to True to make a deep copy of the model

Returns

new model instance

dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

classmethod from_orm(obj: Any) → Model
classmethod get_lc_namespace() → List[str]

Get the namespace of the langchain object.

For example, if the class is langchain.llms.openai.OpenAI, then the namespace is [“langchain”, “llms”, “openai”]

classmethod is_lc_serializable() → bool

Is this class serializable?

json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

classmethod lc_id() → List[str]

A unique identifier for this class for serialization purposes.

The unique identifier is a list of strings that describes the path to the object.

map(run: Run) → Dict[str, str][source]

Maps the Run to a dictionary.

classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model
classmethod parse_obj(obj: Any) → Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode
to_json() → Union[SerializedConstructor, SerializedNotImplemented]
to_json_not_implemented() → SerializedNotImplemented
classmethod update_forward_refs(**localns: Any) → None

Try to update ForwardRefs on fields based on this Model, globalns and localns.

classmethod validate(value: Any) → Model
property lc_attributes: Dict

List of attribute names that should be included in the serialized kwargs.

These attributes must be accepted by the constructor.

property lc_secrets: Dict[str, str]

A map of constructor argument names to secret ids.

For example,

{“openai_api_key”: “OPENAI_API_KEY”}

property output_keys: List[str]

The keys to extract from the run.