The previous post reported the execution result. This companion documents the evaluation contract behind it.
Beating time-weighted average price (TWAP) was not enough. The two-hidden-layer DQN, which maps sixteen market-state features to eight execution actions, also had to beat tuned one-step control without future leakage, unfinished inventory, or feedback from the final seeds.
That required one transition engine, common completion accounting, tuned baselines, role-separated seeds, and frozen checkpoints. The implementation, manifest, and per-seed outputs live in egpivo/amm-lab under data/rl_equilibrium/. The simulator and result are in the previous post; this post covers the five checks behind them.
The learner and baselines must face the same transition law, and the observation cannot contain future information.
Market physics live in one Rust execution environment (). Python never reimplements the step. The DQN calls the same binary over a JSON-lines bridge; every heuristic and planner calls the same step function.
Decision-time fees are snapshotted before any leg runs. Fees update only after the oracle advances. Intra-step priority -whether the agent’s trade runs before or after noise flow and arbitrage within the step-is a declared parameter, not an implementation accident.
A cargo test whitelists every field on the observation vector. Add a field without review and the test fails. Same seed → identical episode outcome. Drift + slippage-ex-fee + fee + gas + terminal term must equal reported implementation shortfall to 1e-6.
The learner also sees less than the tuned baseline: heuristics read the full observation struct; the DQN reads sixteen scaled features. The information asymmetry runs against the learner.
Rust keeps simulation deterministic and semantically single. Python keeps network, optimizer, reward-scaling, and exploration changes cheap.
The PyTorch path is rl_equilibrium_bridge → → dqn_train.py. Tabular Monte Carlo stays in Rust, where thousands of episodes finish in seconds. Rewriting or in Python would duplicate the transition law and create another place for train/evaluation drift.
rl_equilibrium_bridge is a thin stdin/stdout JSON layer. Rust holds state and computes reward; Python sends reset and step, then reads observation, reward, and completion. Tabular runs never cross the bridge.
The behavior is economically legible, though this diagnostic does not establish optimality.
TWAP, myopic routing, and an untuned heuristic are weak comparison points.
The bar is tuned one-step lookahead. It rebuilds exact quote curves and scores each action using immediate execution premium plus a carry for unfinished inventory. The carry multiplier is selected on validation seeds; the frozen value is 16. Immediate liquidation and myopic routing are still reported.
On the final block under agent-first ordering, paired differences relative to lookahead are +10.8 bps for two-step expectimax, +1.6 for three-step expectimax, +0.1 for stochastic rollout, and −14.9 for the DQN (final_ladder.csv, m3r_stochastic_planner_final.csv). Positive values are worse; negative values are better. Stochastic rollout ties lookahead, while the DQN reduces shortfall. Realized fee and quote variation may contribute, but the experiment cannot separate that mechanism from shallow search and heuristic continuation.
The learner rung also includes model-free tabular Monte Carlo control at two discretizations. The fine table ties tuned lookahead; the coarse table does not. That comparison is consistent with a discretization bottleneck in the coarse learner rather than an absence of sequential structure.
Hindsight coordinate descent over full action sequences provides an achieved reference only. It consumes realized future shocks and is never a deployable policy.
An execution policy should not win by leaving inventory unfinished.
Under forced terminal completion-the headline setting-any inventory left after the agent’s final action is executed at the terminal state. Completion is 1.0 for every policy. No controller wins by leaving the book open.
Reward is the negative normalized execution premium: maximizing undiscounted return is identical to minimizing reported implementation shortfall. The terminal term is part of the same accounting identity: a penalty under the standard rule, and actual forced execution cost under the headline rule. It is never removed from reported shortfall or added as a post-hoc correction.
Seeds used for training, selection, diagnostics, and the headline need distinct roles.
Training, development, fresh-check, and final headline blocks are isolated by seed range. Learner validation ( 20,000-20,049) is declared upfront as a nested subset of the broader baseline-validation block ( 20,000-20,199):
The completed manifest at data/rl_equilibrium/m3r_run_manifest.json records Python/torch/rustc versions, hash, git commit, command list, the pre-final design choices, and sha256 of every checkpoint and result CSV. Because it hashes final-result CSVs, the manifest is assembled after those files are produced. The policies and evaluation choices it records were frozen before the final block. make -C scripts/rl_equilibrium verify runs verify_paper_artifacts.py against the recorded contract.
Headline (final block only): forced completion, agent trade after noise and arbitrage, n = 1,000 — DQN 100.3 bps vs lookahead 113.6 bps, paired −13.29 bps [−14.22, −12.32]. Agent trade before noise and arbitrage: −14.9 bps. Randomized intra-step ordering is the smallest edge: −5.62 bps [−7.03, −4.18].
That ordering spread has a loose analogue in recent centralized-exchange-DEX (CEX-DEX) work on priority fees and stochastic settlement delays ( Bergault, Hafsi & Sánchez-Betancourt, arXiv:2602.10798): execution speed, uncertainty, and cost trade off. It is not the same model as the simulator’s trade-ordering parameter; the basis-point numbers are ours. The paper helps motivate treating intra-step priority as a finance parameter rather than a bookkeeping detail.
Retraining under a perturbation does not show that the original checkpoint generalized.
Perturbations change the transition law. The protocol separates:
On development seeds, moving an agent-first checkpoint to agent-last execution shrinks its paired edge from −15.5 to −6.3 bps (m3r_priority.csv). After retraining under agent-last ordering, the final-block edge is −13.3 bps against matched lookahead (m3r_final_paper_seeds.csv). The first comparison measures transfer; the second re-solves the changed environment.
Scalar nuisance perturbations-gas level, arbitrage speed, noise scale, fee coefficients-are transfer-only. None flip the ranking on the frozen checkpoint.
Fee-mode re-solving (a 300-seed subset of the development-test block, default bridge ordering-agent trade before noise and arbitrage): when trained and evaluated under constant fees, the paired DQN-lookahead estimate is +1.2 bps (means round to 116.1 and 115.0 bps), a near tie rather than evidence of an advantage. Dynamic monopoly: −23.2 bps. Dynamic duopoly: −14.7 bps. Each row pairs the retrained DQN against tuned lookahead in the same fee environment (m3r_dynamic_fee_ablation.csv). The edge concentrates in dynamic-fee environments. These are ablation results, not final-block headline estimates.
Two optional sensitivity layers default off in the headline freeze. On separate 500-seed blocks, neither layer flips the DQN-lookahead ranking at baseline or weak stress (m4_lp_adaptation.csv, m4_jit_mev.csv). Aggressive threshold-based sandwich-searcher stress raises shortfall for both policies but leaves the gap intact.
The contract supports a model-conditioned comparison with tuned one-step routing under forced completion. It does not establish live profitability, actual mempool ordering, adaptive liquidity-provider behavior, or equilibrium of the fee rule.
If I rebuilt this study next, resolving intra-step priority and adaptive liquidity-provider response would take priority over network width.
The result counted only after the transition law, completion rule, baseline, checkpoint, and final seed block stopped moving. The next test is a richer transition law with mempool ordering and adaptive liquidity-provider repositioning before adding network capacity.
This post was originally published on my personal blog: https://egpivo.github.io/2026/08/11/before-the-result-could-count-the-benchmark-had-to-freeze.html.
Before the Result Could Count, the Benchmark Had to Freeze was originally published in Coinmonks on Medium, where people are continuing the conversation by highlighting and responding to this story.
