> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ressl.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# SDK reference

> Reference for EvalWorker, EvalTask, events, and the evaluation API.

```bash theme={null}
npm install @resslai/eval
```

```ts theme={null}
import { EvalWorker, type EvalTask, type McpConfig } from "@resslai/eval";
```

## EvalWorker

```ts theme={null}
await new EvalWorker(options).start();
```

`start()` resolves once this worker's run has no tasks left. Call `stop()` to
finish the current work and stop pulling tasks.

| Option           | Default  | Description                                                        |
| ---------------- | -------- | ------------------------------------------------------------------ |
| `baseUrl`        | required | Ressl API URL, such as `https://simulation.ressl.ai`               |
| `apiKey`         | required | Organization API key beginning with `rsk_`                         |
| `runId`          | required | The run to drain — the `runId` returned when the run was triggered |
| `model`          | required | Model identifier used by your agent                                |
| `runAgent`       | required | Async function that runs your agent for one task                   |
| `concurrency`    | `1`      | Maximum number of tasks to run at the same time                    |
| `pollIntervalMs` | `1000`   | Delay between empty-queue polls, in milliseconds                   |
| `onEvent`        | none     | Callback for worker events                                         |

The `model` value must match the model used inside `runAgent`. Ressl uses this
value to compare runs that use the same dataset and model. The constructor throws
if `model` or `runId` is empty.

## EvalTask

Ressl passes this object to `runAgent`:

```ts theme={null}
interface EvalTask {
  runId: string;
  taskResultId: string;
  taskId: string;
  ticket: string;
  mcp: McpConfig;
}

interface McpConfig {
  type: "http";
  url: string;
  headers?: Record<string, string>;
}
```

| Field          | Description                                        |
| -------------- | -------------------------------------------------- |
| `runId`        | Evaluation run that contains the task              |
| `taskResultId` | Identifier used by the worker to report completion |
| `taskId`       | Task identifier in the dataset                     |
| `ticket`       | Prompt to give your agent                          |
| `mcp`          | MCP server for the task's isolated mock world      |

Add `task.mcp` to the MCP configuration used for that agent call:

```ts theme={null}
mcpServers: {
  evalmock: task.mcp,
}
```

<Tip>
  Tasks use separate mock worlds, so they can run in parallel. If your
  organization reaches its world limit, the worker waits and retries.
</Tip>

## Events

Use `onEvent` to log or monitor worker activity:

| Event      | When                                                |
| ---------- | --------------------------------------------------- |
| `task`     | The worker pulled a task                            |
| `reported` | The worker reported task completion                 |
| `idle`     | The queue was empty on this poll                    |
| `drained`  | All worker loops stopped and `start()` is resolving |
| `error`    | A request failed and may be retried                 |

```ts theme={null}
onEvent: (event, data) => {
  console.log(event, data);
}
```

The `task` event includes `{ runId, taskResultId, taskId }`. The `reported` event
includes `{ taskResultId, error? }`. Error data includes the failed operation and
either an error message or HTTP status.

## Failure behavior

* The worker retries temporary request failures with exponential backoff, up to
  30 seconds between attempts.
* A `401` or `403` response stops the worker.
* The worker tries up to three times to report task completion.
* If `runAgent` throws, the worker reports the error. Ressl still grades the
  state left in the mock world.

## Trigger API

The included [GitHub Actions workflow](/evals/quickstart#4-add-the-workflow)
uses this endpoint to start a run.

```bash theme={null}
POST https://simulation.ressl.ai/api/v1/evals
Authorization: Bearer rsk_...
```

```json theme={null}
{
  "commit": "1234567",
  "dataset": "itbench",
  "ref": "refs/heads/main",
  "tasks": ["task-c2"]
}
```

| Field     | Required | Meaning                                                  |
| --------- | -------- | -------------------------------------------------------- |
| `commit`  | yes      | Git SHA to evaluate                                      |
| `dataset` | yes      | [Dataset slug](/evals/overview#datasets)                 |
| `ref`     | no       | Git ref to display with the run                          |
| `tasks`   | no       | Task IDs to run; omit this field to run the full dataset |

```json theme={null}
{ "runId": "…", "status": "running", "taskCount": 89 }
```

The API key determines the organization. The endpoint returns `404` if the
dataset is unavailable or the task filter has no matches.

## Run status API

Check the run until `status` is `succeeded` or `failed`:

```bash theme={null}
GET https://simulation.ressl.ai/api/v1/evals/{runId}
Authorization: Bearer rsk_...
```

```json theme={null}
{
  "runId": "…",
  "commit": "1234567",
  "dataset": "itbench",
  "model": "claude-sonnet-5",
  "status": "succeeded",
  "resolved": 71,
  "total": 89,
  "assertions_passed": 402,
  "assertions_counted": 431,
  "conclusion": "success",
  "regression": { "baselineCommit": "7ab12c3" },
  "comment_markdown": "## Ressl eval...",
  "tasks": [
    {
      "task": "task-c2",
      "status": "graded",
      "reward": 1,
      "assertions_passed": 6,
      "assertions_counted": 6,
      "error": null
    }
  ]
}
```

| Field                                      | Meaning                                                    |
| ------------------------------------------ | ---------------------------------------------------------- |
| `status`                                   | `running`, `succeeded`, or `failed`                        |
| `conclusion`                               | `success`, `failure`, or `neutral` when no baseline exists |
| `resolved` / `total`                       | Fully resolved tasks and total tasks                       |
| `assertions_passed` / `assertions_counted` | Passed and total assertions                                |
| `regression`                               | Comparison with the baseline run, or `null`                |
| `comment_markdown`                         | Markdown summary used by the CI workflow                   |
| `tasks`                                    | Per-task status, reward, assertions, and error             |