Skip to main content
Ressl evaluates your agent by running a dataset of tasks against fresh mock worlds. It grades the final state of each world, compares the result with a previous run, and reports regressions in CI. You connect your agent through a runAgent(task) function. The @resslai/eval SDK handles the rest: it pulls tasks, provides an MCP server for each task, and reports when the agent finishes.

​
How it works

  1. CI starts an evaluation for the current commit.
  2. Your worker pulls tasks from the evaluation queue.
  3. Each task includes a ticket and an isolated MCP server.
  4. Your agent uses the MCP tools to complete the ticket.
  5. Ressl grades the final state and deletes the mock world.
  6. Ressl compares the run with the previous run for the same dataset and model.
The worker only makes outbound requests. You do not need to expose a server, and your model credentials stay in your environment.

​
What gets measured

Ressl calculates two metrics from the final state of each mock world: The commit check fails if either metric is lower than the previous run for the same dataset and model.
Ressl does not choose or run your model. Set the SDK’s model option to the same model identifier used by your agent. Ressl uses this value to group comparable runs.

​
Datasets

A dataset is a collection of tasks and grading assertions. You can find your datasets in the Datasets tab of the Ressl console. Use the dataset slug when starting a run. You can evaluate the full dataset or provide specific task IDs for a smaller smoke test.

Quickstart

Install the SDK and add evaluations to CI.

SDK reference

Review the worker options, task fields, and events.