# v4: the missing explore cell, and a second model (pre-registered 2026-10-22, before any v4 run)

Why: an outside review of findings v3 said (1) result 3 changed the option's wording and its action together,
so "fix the tool before rewriting its description" is not isolated, and (2) every result is one model.

Fixed: scenario episode1-ablation (new game, 300 moves, $6 cap per run), the commit this file is committed in,
the v3 harness otherwise unchanged. Extended thinking off (PE_THINK=0) unless stated.

Arms:
  X  Haiku 4.5, list, PE_EXPLORE=blind-works: the original "Explore up" label and text, word for word,
     with the walk-to-the-farthest-reachable-tile action behind it. Seeds 101-106.     (completes result 3)
  S1 Sonnet 5.5, assist (pathfinding tools).            Seeds 101-103.   (result 1)
  S2 Sonnet 5.5, list, PE_EXPLORE=blind.                Seeds 101-103.   (results 1, 2 off, 3 "Explore up")
  S3 Sonnet 5.5, list + extended thinking (1,024).      Seeds 101-103.   (result 2 on)
  S4 Sonnet 5.5, list, PE_EXPLORE=frontier.             Seeds 101-103.   (result 3)
Screen-only and map-only arms are skipped for Sonnet to keep cost under $25.

Measures (same as v3): got_starter, visited_oldale, beat_rival (yes/no, move), first entry to the house
next door, explore picks on Route 103, cost.

Reporting rules (decided now):
  - Every run is reported; exclusions only for harness crash or API failure, named on the page.
  - Result 3, wording vs action: compare X with v3's "Explore up" (wording same, action differs) and with
    v3's working option (action same, wording differs). Report counts out of 6.
  - Second model: report Sonnet's counts out of 3 next to Haiku's out of 6. With 3 runs, describe direction
    only ("same direction", "no difference", "reversed"); no claims of size.
  - Result 2 stays descriptive: no causal "because"; quoted thinking only.

## Amendment, 2026-10-22, before any Sonnet result was looked at

Sonnet 5.5 can't run a thinking-off arm: its OpenRouter endpoint answers "Reasoning is mandatory for this
endpoint and cannot be disabled", and with no reasoning setting it thinks when it chooses (the first S2/S4
runs logged thinking text with PE_THINK=0). The four Sonnet 5.5 arms were stopped after about 10 moves per
run and are not used. The second model is Claude Sonnet 5 (anthropic/claude-sonnet-5) instead: tested
2026-10-22, it uses 0 reasoning tokens by default and thinks when given a 1,024-token budget, like Haiku 4.5.
Arms S1-S4 are otherwise unchanged. Arm X (Haiku) was already running and is unaffected.

## Amendment 2, 2026-10-22, before any Sonnet result was looked at

Sonnet 5 doesn't think on these decisions even when asked: on a move-shaped prompt it used 0 reasoning tokens
with a 1,024 or 4,000-token budget and with effort "medium" or "high", and the S3 runs logged no thinking.
So S3 can't test result 2 and was stopped after a few moves (not used). Sonnet 5 stays as the second model
for results 1 and 3 (S1, S2, S4).

Result 2 on a second model uses Claude Sonnet 4.6 (anthropic/claude-sonnet-4.6), which does think when given
a budget (98 reasoning tokens on the same prompt, against 393 for Haiku 4.5):
  T1 Sonnet 4.6, list, PE_EXPLORE=blind, thinking off (PE_THINK=0).   Seeds 101-103.
  T2 Sonnet 4.6, list, PE_EXPLORE=blind, thinking on (1,024).          Seeds 101-103.
Same measures and reporting rules as above.