# Modular Arithmetic Challenge Playground

Competition ID: `modular-arithmetic-challenge`

Use this guide when automating Playground evaluation for a pinned Hugging Face model revision before creating the official model-reference submission.

## Endpoint map

Use `competitionId = modular-arithmetic-challenge`.

| Task | Endpoint | Scope |
| :--- | :--- | :--- |
| Read evaluation problem sets | `GET /api/public/v1/competitions/{competitionId}/playground/problem-sets` | `playground.read` |
| Create a model-reference evaluation run | `POST /api/public/v1/competitions/{competitionId}/playground/runs` | `playground.write` |
| Read runs and case results | Shared Playground run endpoints | `playground.read` |

For the official competition submission, use the competition submission endpoint documented in the [Modular Arithmetic Challenge competition guide](../competitions/modular-arithmetic-challenge.md).

## Run payload

Modular Arithmetic Challenge uses `submissionSpec.kind: "model-reference"`. Playground runs evaluate one pinned Hugging Face model revision against one selected evaluation problem set.

**Example request body**

```json
{
  "modelName": "your-team/your-model",
  "commitHash": "0123456789abcdef0123456789abcdef01234567",
  "problemSetId": "mps_eval_01",
  "hfToken": "hf_...",
  "note": "Candidate revision for hard-tier evaluation."
}
```

Payload fields:

| Field          | Type   | Required | Description                                                 |
| :------------- | :----- | :------: | :---------------------------------------------------------- |
| `modelName`    | string | Yes      | Hugging Face model repository in `<owner>/<name>` form.     |
| `commitHash`   | string | Yes      | Full 40-character lowercase hex commit SHA. Branch names are not accepted. |
| `problemSetId` | string | Yes      | Evaluation problem set ID from the Playground problem sets endpoint. |
| `hfToken`      | string | No       | Hugging Face token for private or gated repositories. The token is never returned by the API. |
| `note`         | string | No       | Private note shown in run history. Maximum `500` characters. |

## Submit a run

```bash
curl -X POST "https://api.sair.foundation/api/public/v1/competitions/modular-arithmetic-challenge/playground/runs" \
  -H "Authorization: Bearer $SAIR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "modelName": "your-team/your-model",
    "commitHash": "0123456789abcdef0123456789abcdef01234567",
    "problemSetId": "mps_eval_01",
    "note": "Candidate revision for hard-tier evaluation."
  }'
```

The response returns a `runId`. Read the run and its results through the shared Playground run endpoints.

## Result model

Completed runs include a model-reference summary with `overallAccuracy`, `highestTierAbove90`, per-tier scores, determinism, artifact size, and inference time. Run results return one row per evaluated case with `caseId`, `tierId`, `prediction`, `correct`, and `expected`.

## Workflow

1. Read the competition detail and confirm `submissionSpec.kind` is `model-reference`.
2. Read the available Playground problem sets and choose an evaluation set.
3. Submit a model-reference Playground run with `modelName`, `commitHash`, and `problemSetId`.
4. Poll the run and read case results until the run reaches a terminal status.
5. Create the official submission with the same `modelName` and `commitHash`.
6. Optionally publish the same model reference through Contributor Network.

See [Playground](../playground.md) for the shared endpoint field tables.