> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ressl.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evals quickstart

> Connect your agent to Ressl and run evaluations in CI.

## 1. Install

```bash theme={null}
npm install @resslai/eval
```

Requires Node 20+.

The example below uses TypeScript. Install `tsx` if your project does not
already have a TypeScript runner:

```bash theme={null}
npm install --save-dev tsx
```

## 2. Create a worker

Create a file that runs your agent for each evaluation task. Pass `task.ticket`
as the prompt and add `task.mcp` to your agent's MCP servers.

This example uses the Claude Agent SDK. The same pattern works with any agent
that accepts an MCP server configuration.

```ts agent.ts theme={null}
import { EvalWorker } from "@resslai/eval";
import { query } from "@anthropic-ai/claude-agent-sdk";

const MODEL = "your-model-id";

await new EvalWorker({
  baseUrl: process.env.RESSL_EVAL_URL!,
  apiKey: process.env.RESSL_KEY!,
  runId: process.env.RESSL_RUN_ID!,
  model: MODEL,
  concurrency: 4,
  runAgent: async (task) => {
    for await (const _ of query({
      prompt: task.ticket,
      options: {
        model: MODEL,
        mcpServers: { evalmock: task.mcp },
        allowedTools: ["mcp__evalmock__*"],
      },
    })) {
      // Consume the stream until the agent finishes.
    }
  },
}).start();
```

Replace `your-model-id` with the model used by your agent. The `model` option is
a label for grouping comparable runs, so it must match the value passed to your
agent.

Add a start script so CI can run the worker:

```json package.json theme={null}
{
  "scripts": {
    "start": "tsx agent.ts"
  }
}
```

<Note>
  `runAgent` does not return a score. Ressl grades the final state of the mock
  world after the function finishes.
</Note>

## 3. Copy the dataset slug

Open the **Datasets** tab in the [console](https://simulation.ressl.ai), pick a
dataset, and click the copy icon next to its name.

Use the copied slug in `RESSL_DATASET`. A display name and slug may be different.

## 4. Add the workflow

The SDK package includes a GitHub Actions workflow. Copy it into your repository:

```bash theme={null}
mkdir -p .github/workflows
cp node_modules/@resslai/eval/ci/ressl-eval.yml .github/workflows/
```

On each push to `main`, the workflow:

1. Starts an evaluation for the current commit.
2. Runs your worker with `npm start`.
3. Waits for the worker to finish the task queue.
4. Posts the result as a commit comment and a GitHub check.

Add these values under **Settings → Secrets and variables → Actions**:

| Name                | Kind     | Value                             |
| ------------------- | -------- | --------------------------------- |
| `RESSL_API_KEY`     | secret   | `rsk_…` from the console          |
| `ANTHROPIC_API_KEY` | secret   | API key used by the example agent |
| `RESSL_EVAL_URL`    | variable | `https://simulation.ressl.ai`     |
| `RESSL_DATASET`     | variable | Dataset slug from step 3          |
| `RESSL_TASKS`       | variable | Optional comma-separated task IDs |

<Note>
  The included workflow is configured for the Claude example above. If your
  agent uses another provider, update the workflow to pass the required
  credentials.
</Note>

<Tip>
  Set `RESSL_TASKS` to a few task IDs for quick checks on each push. Leave it
  empty to run the full dataset.
</Tip>

## 5. Run an evaluation locally

Start an evaluation before running the worker. If the queue is empty, the worker
exits.

```bash theme={null}
curl -sS -X POST https://simulation.ressl.ai/api/v1/evals \
  -H "Authorization: Bearer rsk_..." \
  -H "content-type: application/json" \
  -d '{"commit":"'"$(git rev-parse HEAD)"'","dataset":"itbench","tasks":["task-c2"]}'
```

```json theme={null}
{ "runId": "…", "status": "running", "taskCount": 1 }
```

Then drain it with your worker:

```bash theme={null}
export RESSL_EVAL_URL="https://simulation.ressl.ai"
export RESSL_KEY="rsk_..."
npm start
```

You can follow the run from the **Runs** tab in the console.