Criteria: EvalConfig & {
    evaluatorType: "criteria";
    criteria?: Criteria | Record<string, string>;
    llm?: Toolkit;
}

Configuration to load a "CriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria.

Type declaration

The criteria to use for the evaluator.

The language model to use for the evaluator.

The configuration for the evaluator.

const evalConfig = {
evaluators: [Criteria("helpfulness")],
};
@example
```ts
const evalConfig = {
evaluators: [
Criteria({
"isCompliant": "Does the submission comply with the requirements of XYZ"
})
],
const evalConfig = {
evaluators: [{
evaluatorType: "criteria",
criteria: "helpfulness"
formatEvaluatorInputs: ...
}]
};
const evalConfig = {
evaluators: [{
evaluatorType: "criteria",
criteria: { "isCompliant": "Does the submission comply with the requirements of XYZ" },
formatEvaluatorInputs: ...
}]
""