---
name: gpt-image-test
description: Black-box check of an OpenAI-compatible gpt-image API or relay (中转站). Use when the user wants to verify that an image endpoint (base URL + API key) really generates images, compare relays, or get a shareable report on generation, sizes, formats, transparency, image count and editing. Runs locally, or hosted on SandBase Lab with a public board.
version: 0.8.0
---

# gpt-image relay check

Runs the bundled checker (the `image_checker` Python package in `src/`) against an OpenAI-compatible image endpoint and produces a report with a 0–100 score. Every case is an ordinary `/v1/images/generations` or `/v1/images/edits` request; every returned image is decoded and judged on pixel size, format, alpha channel, image count and whether it is a blank placeholder.

Two ways to run the same checker, with the same cases and scoring:

| | **Local** `scripts/check.sh` | **Hosted** `scripts/check_hosted.py` |
|---|---|---|
| Runs on | This machine | A one-off SandBase sandbox |
| Key | Never leaves this machine | Sealed until the sandbox claims it once; not stored or logged |
| Needs | Python 3.11+ (Pillow auto-installed into `.venv/`) | Python 3.9+, nothing else |
| Output | `report.json` + `report.html` (each case shows the prompt, the reference images sent and the images returned) | Report page link, optional public board listing, badge |
| Version | This folder (re-install from SandBase Lab to update) | Latest release, kept up to date by SandBase |

Default to **local**. Offer **hosted** when the user wants a public, shareable report or board listing, or cannot install Python 3.11.

## 1. Get the endpoint and key safely

Ask for the **base URL** (e.g. `https://api.example-relay.com`; a trailing `/v1` is fine) and **model** (default `gpt-image-2`). Do **not** ask the user to paste the key into the chat. Ask them to create a key file:

```sh
printf 'IMAGE_CHECKER_API_KEY=%s\n' 'sk-...' > .image-checker.env && chmod 600 .image-checker.env
```

`IMAGE_CHECKER_API_KEY` already exported in the environment also works. Recommend a dedicated test key: every case is a real, billed image request on their relay.

Never print, `cat`, echo or log the key, pass it as a command-line argument, or copy it into any other file.

## 2. Pick the tests

| Suite | Cases | Time | Use for |
|---|---|---|---|
| `quick` | 12 | 1–2 min | First look: official protocol, one case per area, every watermark check, and the model-identity probe |
| `full` | 47 | 10–14 min | A complete report to publish or compare |

Or exact cases: `--cases default,size-square,edit-mask`. All cases: `references/cases.md`, or `scripts/check.sh --list --model <id>` (shows how each case is judged for that model).

### Model-aware judging

`gpt-image-2`, `gpt-image-2.5-flare` and `gpt-image-2.5-sunburst` (and their dated snapshots, e.g. `gpt-image-2.5-flare-2026-09-08`) are judged against what the official API accepts for that model (`src/image_checker/models.py`):

| Case | gpt-image-2 | gpt-image-2.5 Flare / Sunburst |
|---|---|---|
| `quality-xhigh`, `quality-max` | must be **rejected** (4xx) | must **render** |
| `edit-fidelity` (`input_fidelity`) | must be **rejected** | must be **rejected** |
| `transparent` | optional (preview) | must render with alpha |

An endpoint that renders xhigh for "gpt-image-2", or rejects it for a 2.5 model, fails with `model_mismatch`: it is probably serving a different model under that name. "Optional" cases that come back unsupported are skipped, not scored. Other model ids keep the original matrix, with the 2.5-only tiers skipped. To test all three models, run one check per model.

### A relay that names the model differently
`--model` is what is sent to the relay, verbatim. If the relay calls the model something else (`my-relay-image-2`), add `--profile gpt-image-2.5-flare` (or `gpt-image-2`, `gpt-image-2.5-sunburst`, `generic`) to judge it as that official model. Without `--profile`, the name decides: the three official ids and their dated snapshots get their profile, anything else runs the generic matrix.
### Relays with non-standard paths or headers
Some relays mount the image API elsewhere or want extra headers. Add `--generations-path` / `--edits-path` (`{model}` is filled in with the model id) and `--header 'Name: value'`. A header that carries a secret (for example `x-api-key`) goes into the env file as `IMAGE_CHECKER_HEADERS={"x-api-key":"sk-..."}`, not on the command line. See `references/cases.md`.
### Watermarks
`watermark-visible` and `watermark-dark` request a plain gradient (pale, then dark, so a white mark shows too) and look for a logo or text on the border of the image; `watermark-metadata` looks for watermark-like text in the file metadata, and `signature-c2pa` notes C2PA content credentials. Every watermark case records the scan of each returned image in `images[].watermark` (detail map) and `images[].metadata` (format, metadata blocks, text fields, C2PA and its signer), and the HTML report and `--progress` print it. They are heuristics: tell the user what was found and where, and ask them to open the image to confirm.
## 3a. Run locally

```sh
<skill-dir>/scripts/check.sh --env-file .image-checker.env \
  --base-url https://api.example-relay.com --model gpt-image-2 \
  --suite quick --title "Example Relay" --out ./image-check-report
```

Writes `report.json` and `report.html` to `--out` and prints one line per finished case. Every case also keeps what the relay answered (`response`), shown under each case in the HTML report. `--lang zh-CN` writes the HTML report and messages in Simplified Chinese (default `en`); match the user's language. `--concurrency 1..8` (default 2); lower it if the relay rate-limits. `--timeout` per request (default 180 s).

## 3b. Run hosted on SandBase Lab

```sh
<skill-dir>/scripts/check_hosted.py --env-file .image-checker.env \
  --base-url https://api.example-relay.com --model gpt-image-2 \
  --suite full --title "Example Relay" --out ./hosted-report [--public] [--mask-url]
```

Prints the report link (`https://www.sandbase.ai/lab/gpt-image-test/r/<id>`) right away, follows progress, then saves `report.json`. `--public` lists the report on the public board; `--mask-url` hides the host (masked reports are never listed). It also saves `owner-token.txt`, which lets the user change visibility later. Details: `references/hosted.md`.

## 4. Report back

From `report.json` (it never contains the key), tell the user:

1. **Score** and **verdict**. If the verdict is `key_rejected`, `model_missing`, `no_credit` or `unavailable`, lead with that and how to fix it; the rest is not meaningful yet.
2. **Per area** (`groups`): which pass fully, which fail.
3. **Protocol conformance**: list `protocol_errors` and `protocol_warnings` from the report (deviations from the official OpenAI Images API spec, e.g. missing `usage`, parameters not echoed).
4. **Each failed case** with its `failure_class` in plain words (`references/failure-classes.md`). Separate `unsupported` (an honest capability gap) from `contract` (an image came back but does not match the request, a strong sign of a downgraded or substituted model), `model_mismatch` (the endpoint behaves like a different model than the one named) and `protocol` (the response breaks the official OpenAI Images API schema, so official SDKs may break). Mention skipped cases and their `skip_reason`.
5. **What was sent** when a case is disputed: every result carries its `prompt`, and edit cases carry the reference images in `inputs` (decoded, with thumbnails), so input and output sit side by side.
6. **Average latency** of passed cases.
7. Where the report is: `report.html`, or the hosted link.

To compare with a known-good endpoint, use the official runs in `results/` (index: `references/official-results.md`).

## Files

- `references/architecture.md`: how checking, scoring and the safety rules work; local vs hosted
- `references/cases.md`: every case, its request and its contract
- `references/failure-classes.md`: what each failure means and what to tell the user
- `references/report-format.md`: `report.json` fields
- `references/hosted.md`: hosted API, visibility, badge
- `references/runtime-setup.md`: requirements, key file, updating, troubleshooting
- `references/official-results.md`: official reference runs
- `scripts/check.sh`: local run · `scripts/check_hosted.py`: hosted run · `scripts/render_report.py`: JSON → HTML (`--lang en|zh-CN`)
- `results/`: official run reports (JSON + HTML)