gpt-5.4-mini, production settings
Claude
Agent
Language
Conversation
Tools the model sees
Next move, repeated
The model gives different answers to the same call. Try ×10 runs the agent's next reply ten times so you see how often each outcome happens.
Use this call in a fix
What should the agent have done or said instead? Try ×10 grades replies against this.
Add it as an example to
Calls
Working hours
Language
Dialect
Transfers to
Also has
What an agent with these settings is told. Company details are a neutral sample.
Pick a test
What this page shows
  • Every test Claude ran for the fix picked at the top: first the fix's own tests, then the wider checks.
  • Before is today's prompt, after is the prompt with the change. A test runs many times because answers vary, so 18/20 means 18 good runs out of 20.
  • Green means the change made it better, red means worse. Pick a test to see what it checks and the actual conversations, failures first.

Waiting for Saturday's merge

Approved fixes. They aren't live yet, so their author can still recall or drop them.

People and their fixes

Follow anyone's fix to watch Claude work on it. Only its author can change it.

Weekly merges

Every Saturday the approved fixes are combined, tested together, and handed to the lead to go live.

How often the grader is right

For developers: working from a terminal (not needed for fixes on this site)
Everything on this site works without this. Each person also has a copy of the code in a folder on the server, kept on the live prompt; it powers "Live prompt" under Testing with. Developers can open it, or start another one for a task, with Claude Code in a terminal (prompt-task).

Prompt Lab

Pick your name to sign in.