StressingLLMs OpenCode Benchmark Results

Models analyze generated binaries whose deterministic state transformations become more complex as the round count increases. Results show observed reach, runtime, token use, cost, and exact grading evidence. Some frontier-model campaigns ended because they were expensive, and I’d rather spend the money elsewhere. Their capability boundaries are incomplete, but estimates are provided.

RECOVERY PIPELINECOMPILED ELF → STANDALONE PYTHON

Each fixture embeds a known plaintext behind deterministic state mixing and XOR decryption. For example, fx_r0068_sl0016_sp0000_dwarf requires the model to understand 68 transformation rounds and reproduce the recovery logic—not merely guess the message.

01open ELF in Ghidra // inspect functions, data, and control flow

02trace 68 mixing rounds // recover state updates and XOR key stream

03write standalone Python // reproduce decryption outside Ghidra

04execute in sandbox // compare output with expected plaintext

> Hello, World

A pass requires executable submitted Python that prints the exact expected plaintext. Tool commentary or an unevaluated answer alone does not pass.

Exploratory use: The author created this page to examine campaign data and identify patterns. Its comparisons and estimates should not be treated as absolute model rankings or definitive capability measurements.

MODEL

Observed reach

Highest verified round in each campaign—not necessarily a capability limit.

COMPLEXITY ↑

Cost-limited campaigns do not establish a boundary. A higher pass also does not imply every lower round passed. Select a bar to inspect its result.

Outcome breakdown

Current attempts in this selection.

01 / COMPLEXITY

The round map

Rounds run downward; models run across. Select a block to open its evidence.

PassFailProvider errorCost-cancelledCost-skippedIncomplete

Time under stress

Recorded attempt runtime as round complexity increases.

SELECT A POINT

Points retain their actual round values; gaps are untested complexities. Runtime is per attempt, in minutes.

Cost under stress

Cost reported by OpenCode's completed model steps as round complexity increases.

SELECT A POINT

Lines connect each paid model's attempts in round order. Every point is one attempt's cost—not cumulative spend. Free-tier attempts are omitted instead of forming a $0 baseline.

Estimated single-attempt resources

Assuming each model continues completing attempts without failing, compare estimated runtime, workflow tokens, or OpenCode cost for one fixture.

METRIC

02 / EVIDENCE

Attempt explorer

Submissions, execution traces, token usage, and the records behind each result.

03 / READING THE RESULTS

What this benchmark measures

01 Recovery, end to end

Generated C fixtures mix state through deterministic transformation rounds and XOR-decrypt a message. A model uses Ghidra tools and submits Python. The grader runs fenced code in a no-network sandbox; any executed block printing the expected plaintext yields a pass. This measures the model and its tool workflow together.

02 Adaptive search, bounded evidence

Most campaigns begin at rounds 1–10, then advance by 2 for ten values, then by 3, and so on; cost-aware campaigns may start later. Two consecutive failures trigger integer bisection and can establish a boundary. Runs stopped for cost report only their highest observed pass and must not be read as finding the model's limit.

03 Costs & interrupted work

Costs sum the values emitted by OpenCode in each attempt's completed model-step events. They are closer to provider charges than configured token-price estimates, but may not equal an account balance change if ancillary requests are not part of the stored attempt transcript. Provider errors and interrupted reservations are excluded from graded pass rates.

Campaign settings & boundary history

Created with OpenAI Codex · GPT-5 family · Codex CLI 0.153.2