evals
Reference for the evals tool: create evals with cases and a scoring method, read them and their past runs.
An eval is a list of cases (an id, an input and what you expect) and a method that scores answers from 0 to 1. A threshold, 0.8 by default, says whether a version passes.
Call it as geenius_promptverse(tool: "evals", action: "<action>", args: {...}) through Geenius OS, or as the evals tool of the Prompt Verse MCP server with action and args.
Actions#
| Action | What it does |
|---|---|
create | A new eval with its cases and method. |
get | One eval with its cases. Your AI can use these cases to test a prompt itself. |
list | Your evals, newest first. |
run | Score a prompt on a model with the saved cases. Switched off in the pre-beta. |
runs | Past runs of an eval. |
Arguments we have checked#
evals · create#
A new eval with its cases and method.
entity_type(required): prompt, workflow or agent.entity_id(required): Its id.title(required): What it checks.cases(required): A list of cases: objects with id, input and expected.method(required): exact, contains, structural, judge or hybrid.threshold: From 0 to 1. Default 0.8.
evals · get#
One eval with its cases. Your AI can use these cases to test a prompt itself.
id(required): Eval id.
evals · run#
Score a prompt on a model with the saved cases. Switched off in the pre-beta.
eval_id(required): Eval id.prompt_id(required): Prompt id.model_id(required): Model id.
evals · runs#
Past runs of an eval.
eval_id(required): Eval id.limit: Default 20.