# The next design-model batch

Six fresh chats. Two prompts. Rate the whole set afterward.

This batch adds Planning room (design a dense working interface) and Review room · follow the plan (execute a specified design). Run each task once in ChatGPT Astra, Claude Fable, and Grok. The three earlier Review room outputs remain pilot results. The new prompts include a run report and therefore belong to protocol v2, not v1.

## Run all six

1. Open six fresh chats: two in each app. Select the model and reasoning setting you intend to compare. Record the exact label shown in the app; do not rely on what the model calls itself.
2. Turn off tools, browsing, memory, custom instructions, and artifact execution where possible. Record any setting you cannot disable. Do not paste these prompts into a chat that has seen earlier results.
3. Paste `planning-room-v2.txt` unchanged into one chat per app. Paste `review-room-plan-v2.txt` unchanged into the other three. Each file is a complete prompt; do not add a provider-specific introduction or attach the other output.
4. You can submit the six chats in parallel. Start a separate stopwatch for each Send and stop it when that answer finishes, including the report. Record concurrent runs in the log: parallel use may affect service limits and timing. Cap each attempt at 15 minutes. Keep partial answers, refusals, outages, and unwanted tool use; do not regenerate a weak answer.
5. Save each entire answer, including its two source blocks and `run-report.json`, as a `.txt` file. Use the six filenames in `run-log.csv`. Do not repair code or rewrite missing metadata. A response without a report is still a result to keep.
6. Return the six answers and `run-log.csv` together. There is no need to inspect or rate each one as it arrives. Leave timings blank if you missed the stopwatch; the model is explicitly told not to invent them.

| App | First chat | Second chat |
| --- | --- | --- |
| ChatGPT Astra | Planning room | Review room · follow the plan |
| Claude Fable | Planning room | Review room · follow the plan |
| Grok | Planning room | Review room · follow the plan |

Only the filenames differ by provider. The prompt for a given task is identical in all three apps. The supplied images are added to the common renderer later; none of these code-only prompts includes a visual image preview.

## What the model reports

Each prompt requires a final JSON block with model identity, reasoning setting, time, token counts, tool use, and limitations. It must use null for anything it cannot observe. A guessed version or duration is not a measurement.

Keep these sources separate:

- **App label:** what you actually selected and can see in the UI.
- **Model self-report:** what the response claims its runtime exposed; unverified until corroborated.
- **Operator time:** Send until the complete answer, including its report.
- **Runtime time:** only when real telemetry exists, with its exact scope. The requested model report ends its timing at the last code block, so it is not the same interval as the operator's stopwatch.

Never replace a missing stopwatch value with a plausible model claim. A null value is useful: it tells us what was not measured.

## Review everything in one session

After all six arrive, the review set will contain nine outputs grouped by their assignment. Generated code must be inspected before it is run. We will use the common React 19.2.4 / Vlak 0.4.0 environment, block external requests, check behavior, and prepare anonymous preview labels. The v1 checker is not valid for Planning room; task-specific checks must be prepared and versioned before scoring the new batch.

The old and new results stay separate in the data. New instructions, a detailed plan, and a different task make this a broader pilot, not six more repeats of the original experiment. One attempt per model per task cannot establish reliability. Do not average quality or speed across the three assignments into a league table.

Set aside about 45–60 minutes. Review one task group at a time, with model names and timings hidden. Keep your notes before revealing the mapping. The first three apps have already been shown with names, so mark prior exposure rather than calling those reviews fully blind. A second reviewer can provide an independent set of ratings.

### The task cards

**Original Review room:** open Field notes; request changes with empty feedback; add a comment; approve another image; reload; search for a missing item and clear the filters. Check the review on a phone, with the keyboard, and in dark mode.

**Planning room:** identify the initial two-hour capacity overrun; move an unblocked task to Later and check the total; try to schedule a blocked task; save an empty next action; edit a real note and reload; combine search and state filters; clear an empty search. Repeat the core edit on a phone and with the keyboard.

**Review room · follow the plan:** complete the review tasks above, then compare with the supplied layout plan. Is the image larger when opened? Do the columns and controls adapt as specified? Keep plan-following defects separate from your personal preference for another layout.

For each app, score the five criteria from 0 to 5 and give an example from the screen. Add the first change you would ask for. You do not need to inspect TypeScript or calculate totals. Technical checks are recorded separately; the weighted design score comes from your ratings.

Use `review-sheet.csv` after the anonymous previews are ready. A blank score means not rated, never zero. The weights are in `protocol.json`. Save ratings locally and submit them as a batch; they will feed the article only after review.

## Files and reproducibility

`manifest.json` records hashes of the prompts and supplied fixture. Both prompts return only App.tsx, app.css, and a factual report. The report is data, not a script. Never install or execute commands supplied by a model.

The fixture and six photographs match the original test environment. The photographs are supplied for this test, not for redistribution. This pack is ready to run manually in the apps; it does not send prompts or publish results on its own.