ISO: An RLVR-Native Optimization Stack

Spectra stay: spectral inheritance in RLVR

Write each weight matrix as W = U Σ V^⊤: the spectrum Σ sets the scale of each singular mode, the frames (U, V) set its output/input directions. Across RLVR runs — including a public reasoning run trained for 3,000+ updates — the learned checkpoint sits strikingly close to the fixed-spectrum family of its base weights: the pooled spectral residual is approximately 3% of the total base-to-RL weight displacement. In the sequential SFT and RLVR comparison, the mean relative residuals are approximately 32% and 2%, respectively. These transitions start from different checkpoints; the SFT endpoint is the RLVR base.

Dimension-normalized spectrum-changing energy remains order-one relative to an isotropic reference for RLVR, so proximity alone does not establish a strong preference for spectrum-preserving updates. We test spectral reuse functionally: restore the base spectrum after training, or never let it move at all.

The rebase experiment, interactive

Take an RLVR checkpoint (DeepSeek-R1-Distill-Qwen-1.5B → Nemotron-Research-Reasoning-Qwen-1.5B), keep its learned frames (U_RL, V_RL) fixed, and interpolate only the spectrum between the base model's Σ₀ and the RL-trained Σ_RL. At α = 0 the RL model's spectrum is fully replaced by the base spectrum. Drag the slider to inspect the relatively small changes in accuracy:

Restoring the base spectrum preserves most acquired RLVR gains. A spectrum-only parameterization (frames frozen, singular values trainable) likewise fails to learn comparably — what RLVR needs to change is the frames.

The mirror test: graft the RL spectrum into the SFT frames

Run the same interpolation in the opposite direction: fix the frames of the SFT model (U_SFT, V_SFT) and blend the spectrum toward the RL-trained Σ_RL. If RLVR's gains lived in the spectrum, scores should climb toward the RL checkpoint as α grows. Scores remain nearly unchanged:

Substituting the RL spectrum into SFT frames does not yield comparable gains. Across the evaluated interpolation coefficients, the seven metrics remain near the SFT model's performance, with an observed range of approximately 3.2 percentage points or less per metric. Together with spectrum restoration in RL-trained frames, this supports the functional reusability of the base spectrum.

Frames move — and both must stay adaptable

If the spectrum can be inherited, which variables must remain free? We reconstruct RL endpoints under four structural restrictions and measure how much of the checkpoint update each one fails to explain (two-stage VLA RL pipeline, transition W₀ → W₂, median across layers):

ISO: Isospectral Optimization

Inherit the spectrum, optimize the frames:  W(U, V) = U Σ₀ V^⊤,  U, V on the Stiefel manifold.

The base optimizer receives ambient factor gradients and takes a tentative frame update that need not lie in the tangent plane. Polar restoration restores orthonormal frames, so W_t = U_t Σ₀ V_t^⊤ retains the base spectrum in exact arithmetic. The ellipsoids and update paths are schematic; the gray planes show local tangent geometry.

Every represented matrix shares the base spectrum Σ₀ exactly; all learning and composition happen in the frame coordinates, with a polar retraction restoring orthonormality. ISO is a stack, not a single optimizer — one principle, two instantiations spanning the RLVR workflow:

ISO-Merger: data-free expert composition

Merging RLVR specialists in fixed-spectrum frame coordinates recovers their complementary capabilities — without post-merge training, rollouts, or distillation. Recovery of each specialist's own score by the single merged model:

Qwen2.5-7B-Instruct · coding + tool-use + memory experts

Means over 3 independent stochastic evaluation runs (standard deviations in the paper). Expert rows shaded blue; bold marks the best expert and the best merged result per column.

DeepSeek-R1-Distill-Qwen-1.5B · coding + math experts

ISO-Merger achieves the strongest aggregate performance among the compared data-free merging methods on both backbones, and improves worst@4 by 1.62 / 1.36 points — more consistent capability recovery, not just a better mean.

Citation

@article{zhu2026iso,
  title         = {ISO: An RLVR-Native Optimization Stack},
  author        = {Zhu, Hanqing and Cong, Wenyan and Sha, Zhizhou and Mukherjee, Sagnik and
                   Song, Xinyuan and Gonz{\'a}lez-Mart{\'i}nez, David and Wu, Xiaoxia and
                   Tian, Yuandong and Liu, Shiwei and Pan, David Z. and Wang, Zhangyang},
  journal       = {arXiv preprint arXiv:2607.19331},
  year          = {2026},
  eprint        = {2607.19331},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/2607.19331}
}