ISO: An RLVR-Native Optimization Stack

Spectra stay: spectral inheritance in RLVR

Write each weight matrix as W = U Σ V^⊤: the spectrum Σ sets the scale of each singular mode, the frames (U, V) set its output/input directions. Across RLVR runs — including a public reasoning run trained for 3,000+ updates — the learned checkpoint sits strikingly close to the fixed-spectrum family of its base weights: the spectral residual is only ≈3% of the total weight displacement, versus ≈35% for a comparable SFT transition. SFT reshapes the spectrum; RLVR doesn't need to.

Proximity alone could be a high-dimensional accident, so we test it functionally, with interventions in both directions: restore the base spectrum after training, or never let it move at all.

The rebase experiment, interactive

Take an RLVR checkpoint (DeepSeek-R1-Distill-Qwen-1.5B → Nemotron-Research-Reasoning-Qwen-1.5B), keep its learned frames (U_RL, V_RL) fixed, and interpolate only the spectrum between the base model's Σ₀ and the RL-trained Σ_RL. At α = 0 the RL model's spectrum is fully replaced by the base spectrum. Drag the slider — the accuracy doesn't care:

Restoring the base spectrum preserves RLVR's gains at every α. A spectrum-only parameterization (frames frozen, singular values trainable) likewise fails to learn comparably — what RLVR needs to change is the frames.

The mirror test: graft the RL spectrum into the SFT frames

Run the same interpolation in the opposite direction: fix the frames of the SFT model (U_SFT, V_SFT) and blend the spectrum toward the RL-trained Σ_RL. If RLVR's gains lived in the spectrum, scores should climb toward the RL checkpoint as α grows. They don't move at all:

The RL spectrum in SFT frames buys nothing — at any mixing ratio. All 7 metrics stay pinned at the SFT level (α-sweep range ≤ 3.2pp, within Mean@n sampling noise; the α=0 endpoint reproduces the original SFT model to ≤ 0.8pp). The spectra themselves barely differ — relative drift ‖ΔΣ‖/‖Σ‖ ≤ 0.04% across all 198 matrices. Together with the panel above, the evidence runs in both directions: useful RLVR progress is carried by singular-frame motion, not by spectral rescaling.

Frames move — and both must stay adaptable

If the spectrum can be inherited, which variables must remain free? We reconstruct RL endpoints under four structural restrictions and measure how much of the checkpoint update each one fails to explain (two-stage VLA RL pipeline, transition W₀ → W₂, median across layers):

ISO: Isospectral Optimization

Inherit the spectrum, optimize the frames:  W(U, V) = U Σ₀ V^⊤,  U, V on the Stiefel manifold.

The tangent space T_Ut rides along the trajectory: at each step the base optimizer takes a tentative update in the tangent plane, and the polar retraction puts the iterate back on the manifold — so every W_t = U_t Σ₀ V_t^⊤ keeps the base spectrum exactly.

Every represented matrix shares the base spectrum Σ₀ exactly; all learning and composition happen in the frame coordinates, with a polar retraction restoring orthonormality. ISO is a stack, not a single optimizer — one principle, two instantiations spanning the RLVR workflow:

ISO-Merger: data-free expert composition

Merging RLVR specialists in fixed-spectrum frame coordinates recovers their complementary capabilities — with zero post-merge compute. Recovery of each specialist's own score by the single merged model:

Qwen2.5-7B-Instruct · coding + tool-use + memory experts

Means over 3 independent stochastic evaluation runs (standard deviations in the paper). Expert rows shaded blue; bold marks the best expert and the best merged result per column.

DeepSeek-R1-Distill-Qwen-1.5B · coding + math experts

ISO-Merger achieves the strongest aggregate performance among the compared data-free merging methods on both backbones, and improves worst@4 by 1.62 / 1.36 points — more consistent capability recovery, not just a better mean.

Citation

@article{zhu2026iso,
  title         = {ISO: An RLVR-Native Optimization Stack},
  author        = {Zhu, Hanqing and Cong, Wenyan and Sha, Zhizhou and Mukherjee, Sagnik and
                   Song, Xinyuan and Gonz{\'a}lez-Mart{\'i}nez, David and Wu, Xiaoxia and
                   Tian, Yuandong and Liu, Shiwei and Pan, David Z. and Wang, Zhangyang},
  journal       = {arXiv preprint arXiv:2607.19331},
  year          = {2026},
  eprint        = {2607.19331},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2607.19331}
}