The rubric rule is used to evaluate contents against a set of predefined criteria or guidelines. This rule is particularly useful for assessing the quality, relevance, or appropriateness of the AI’s responses based on specific requirements or standards.

​
Parameters:

  • type: rubric (specifies the rule type)
  • value: The evaluation criteria or questions
  • expected: Usually set to pass to ensure the content meets the specified criteria
  • threshold: A number between 0 and 1 representing the minimum score required to pass the evaluation (e.g., 0.8 for 80% compliance)

​
Rubric criteria examples

  • “The response must be in English”
  • “Check if the response is a valid JSON”
  • “The user question must be applicable to the assistant propose”
  • “Must accept the negotiation and payment terms”
  • “Do not suggest, calcule or accept any discount”