Replacing subjective human preference signals with objective verifier outcomes like code passing, unit test, or math solutions matching ground truth enables models to optimize directly for correctness and complex reasoning. The shift has delivered massive capability gains at lower cost compared to prior paradigms, making it particularly effective for math coding in a genetic task where reliable feedback loops exist. For more resources on RLVR:

• Daniel Kang's Substack: it talks about enterprise value and first generalization bounds at scale
• DeepSeek-R1 paper: fundamental for modern reasoning models

RLVR#

Algorithm#

Good Verifiers and Data Curation#

Verifier/env packages#

Code Packages#

Literature Review#

ReViSQL#

Existing agents are. compensating for weak SQL reasoning and increasingly complex pipelines. Text-to-SQL has an executable verifier, so the better strategy may be to improve the model's reasoning capabilities through RLVR. However, the benchmarks contain corrupted reward signals that need to be revised. This was central idea behind ReViSQL. The paper broke this down by:

• Diagnosing the bottleneck as model reasoning and not missing pipeline components
• Noticed that [[#Text-to-SQL compatible with RLVR|Text-to-SQ

  

  L is naturally compatible with RLVR]]
• Prior Results w/ RLVR suggest that RLVR already worked for SQL ReViSQL address the benchmark issues and the gold queries in the dataset because more structurally complex (table increased by $12\%$, functions by $10.4\%$, set operations by $\sim 90\%$, subqueries by $66.8\%$, aggregations by $21.9\%$)

Pipeline Engineering is not the bottleneck#

Previous high scoring systems combined ideas like:

• Schema linking or schema pruning to identify the tables, columns, and values relevant to the question [1].
• Question rewriting to clarify ambiguous, incomplete, or schema-inconsistent natural-language questions [2].
• Few-shot retrieval to retrieve similar question–SQL examples and insert them into the model’s prompt [3].
• Query decomposition to divide a complex question into smaller subqueries or reasoning steps [4].
• Candidate generation to produce multiple diverse SQL queries rather than relying on a single generation [4, 5].
• Iterative execution and repair to execute an initial SQL query, observe errors or suspicious results, and repeatedly revise it [6].
• Reward-model ranking to score generated candidates and select the query most likely to be correct [7].
• Majority voting or self-consistency to select the most frequent execution result among independently generated candidates [6, 8].
• Multi-agent orchestration in which specialized agents separately perform schema selection, decomposition, generation, validation, and correction [1, 9]. Combing components can improve accuracy but introduces two problems.
• Cascading errors: If an early schema-linking, question-interpretation or decomposition module makes a error, later agents receive incorrect context and propagates the error into the final SQL query. OpenSearch-SQL explicitly observes that error introduced when an earlier agent persist through subsequent stages, while ReVisSQL identifies this error propagation as a central weakness of multi-stage pipelines [3, 10].
• Inference overhead: Every retrieval operation, specialized agent, candidate-generation call, execution round, and refinement step consumes additional tokens and compute. Candidate-heavy systems can require tens or hundreds of generations for one question, substantially increasing latency and cost [7, 10]. ReViSQL argues that adding another pipeline component would continue patching weakness in the underlying model. The fundamental bottleneck was that the model itself had not learned strong SQL reasoning. This motivated the authors to use RLVR to directly improve the model's ability to perform schema exploration, logical reasoning, query construction, and self correction [10]. The reported SOTA remained approximately $11\%$ points below BIRD's proxy human score.

Text-to-SQL compatible with RLVR#

Text-to-SQL naturally provides what RLVR needs. The evaluator can execute candidate SQL query and evaluate programmatically [10, 12, 13, 14, 15]. For a question $x$, ReViSQL samples a rollout containing reasoning, intermediate database interactions and a final SQL query [10]. The final query is executed and compared with verified gold query results using the assigned grading method:

• $1$, the final SQL passes the assigned execution-based grader
• $0$, the query fails or produces an incorrect result
• $-1$, no final SQL query is produced The reward is supplied programmatically with less (bias and errors) compared to an LLM judge [10, 12]. The model can issue intermediate queries to explore database, test assumptions, and refine its answer. Human-annotated reasoning traces are not required because supervision comes from the final execution result. The model is rewarded for the final outcome instead of being required to imitate one prescribed reasoning trajectories. Compared with single-reference SFT, RLVR can reinforce any sampled trajectory that produces an accepted results, while SFT ordinarily maximizes the likelihood of the provided SQL token sequence [12-15]. However, ReViSQL does not have a RLVR vs. SFT experiment using the same base model and verified data.

Prior Results w/ RLVR#

Along with the compatible with RLVR, prior results have shown that reinforcement learning have strong results.

• Databricks RLVR method reports $75.7\%$ on BIRD
• SQL-R1 trained using a large synthetic dataset reached $66.6\%$
• 600-example experiment provided an initial proof of concept for applying RLVR to corrected human data.

Benchmark Issues in BIRD (correct by ReViSQL)#

Internal inconsistencies: The question, database schema, external knowledge, and gold SQL sometimes disagreed. For example,

• Question: "at least 10 movies"
• External Knowledge: count(direct_name) > 10
• Correct Interpretation: count(direct_name) >= 10

Ambiguous questions: Some questions support several valid interpretations. the benchmark rewards only the annotator's preferred interpretation. This creates one-to-many semantic mappings ($x\rightarrow \{y^{\star}_1, y^{\star}_2, ...\}$) while the benchmark treats it as $x\rightarrow y^{\star}_1$ . RLVR gives false-negative rewards to other reasonable interpretations.

Incorrect gold SQL/ground-truth: Some gold queries simply failed to implement the requested operation.

• Incorrect model SQL can match the faulty gold and receive $+1$
• Correct model SQL can disagree with the faulty gold and receive $0$.

Missing-value errors: Aggregations sometimes included missing or invalid data. For example, a BMI calculation failed to remove:

• NULL heights or weights
• $0$ heights A syntactically valid query could therefore produce a semantically invalid statistics

There were many other issues that includes: failure to account for ties, failure to dedup joined entities, empty answer artifacts. One important issues is that the standard BIRD set-based grader was not appropriate for every questions. ReViSQL added:

• Subset-based grading: for questions asking for "any two" valid answers
• List-based grading: for questions where order matters, such as the top-five products in descending order
• Set-based grading: for cases where row order does not matter Without these distinction, a correct ordered answer can be treated incorrectly or a wrong ordering could potentially receive credit.

After correcting, the gold queries became structurally more complex, suggesting original annotations used simpler SQL by ignoring real constraints. They verified the corrections by the following workflow

The LLM reviewer had $90.6\%$ precision and $24.5\%$ recall. Thus, recall shows that it missed three quarters of the actual problems. Additionally, $16.2\%$ of the first round human corrections failed independent verification and some required up to four rounds.

Experiment w/ Verified Benchmark#

The most important experiment compares the same RLVR method under two data conditions

• original BIRD Train: approximately 9,000 noisy examples
• BIRD-Verified: only 2,462 corrected examples On the verified evaluation data, the verified training had higher test accuracy. Using RLVR also increased observed training reward. The noisy-data model was successfully optimizing its specified reward. The problem was that its specified reward did not correspond to real SQL correctness. This is essentially a corrupted-reward or Goodhart-style failure.

Designing Systems Around RLVR#

The framework to use when determining if RLVR is appropriate for another benchmark is:

• Define the actual capability. Do not assume the current benchmark score measures it.
• Construct an executable or deterministic verifier.
• Check that the verifier accepts multiple valid outputs.
• Audit questions, context, ground truth, and grading logic independently.
• Remove unanswerable and non-discriminating instances.
• Verify training and validation data, not just the test set.
• Train on noisy and verified data with the same RLVR setup.
• Plot training reward against verified held-out accuracy.
• Look for reward increasing while real accuracy decreases.
• Test on an out-of-distribution benchmark.
• Separate single-generation improvement from inference-time scaling.
• Report where human parity comes from: training, verifier quality, selection, or compute. RLVR is extremely powerful when a task has an outcome verifier, but that power makes benchmark errors more dangerous. The optimizer will exploit whatever definition of success it receives, even when that definition is wrong. ReViSQL succeeded because the authors treated verifier and dataset correctness as prerequisites for RLVR, not as cleanup performed afterward.

Training System#

References#

[1] S. Talaei et al., “CHESS: Contextual Harnessing for Efficient SQL Synthesis,” 2024.
Used for: schema linking/schema pruning, candidate generation, iterative refinement, validation, and multi-agent specialization.

[2] P. Ma et al., “A Plug-and-Play Natural Language Rewriter for Natural Language to SQL,” 2024.
Used for: detecting and rewriting ambiguous or flawed natural-language questions. This is a direct supporting example, although it is not cited in ReViSQL’s bibliography.

[3] X. Xie et al., “OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-Shot and Consistency Alignment,” Proceedings of the ACM on Management of Data, 2025.
Used for: dynamic few-shot retrieval, schema extraction, candidate generation, refinement, voting, and cascading-error analysis.

[4] M. Pourreza et al., “CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL,” ICLR, 2025.
Used for: divide-and-conquer query decomposition, multi-path candidate generation, tailored few-shot examples, and learned candidate selection.

[5] Y. Liu et al., “XiYan-SQL: A Novel Multi-Generator Framework for Text-to-SQL,” IEEE Transactions on Knowledge and Data Engineering, 2026.
Used for: schema filtering, diverse candidate generation, and selection-model-based aggregation.

[6] M. Deng et al., “ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration,” 2025.
Used for: execution-guided exploration, iterative syntax and semantic repair, schema linking, and majority-vote consensus.

[7] S. Agrawal and T. Nguyen, “Open-Sourcing the Best Local Text-to-SQL System,” Contextual AI Technical Report, 2025.
Used for: large-scale parallel candidate generation, execution filtering, majority voting, and reward-model ranking.

[8] L. Sheng and S.-S. Xu, “CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning,” 2025.
Used for: parallel sampling, frequency-based voting, candidate merging, and corrective revision.

[9] B. Wang et al., “MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL,” COLING, 2025.
Used for: multi-agent orchestration involving schema selection, decomposition, generation, and SQL refinement.

[10] Y. Zhu et al., “ReViSQL: Achieving Human-Level Text-to-SQL,” 2026.
Used for: the overall argument concerning cascading errors, inference overhead, and limited intrinsic SQL reasoning as the primary bottleneck.

[11] J. Li et al., “Can LLM Already Serve as a Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs,” NeurIPS, 2023.
Used for: the BIRD benchmark and execution-based evaluation of generated SQL queries.

[12] Z. Yao et al., “Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL,” 2025.
Used for: execution correctness as a programmatic reward, outcome-based training, and avoiding brittle intermediate supervision.

[13] M. Pourreza et al., “Reasoning-SQL: Reinforcement Learning with SQL-Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL,” 2025.
Used for: reward-driven SQL reasoning, SQL-specific partial rewards, and comparison between reinforcement learning and supervised fine-tuning.

[14] S. Papicchio et al., “Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQL,” 2025.
Used for: execution-computed rewards and comparisons among zero-shot reasoning, reasoning-aware SFT, and RLVR.

[15] P. Ma et al., “SQL-R1: Training Natural Language to SQL Reasoning Model by Reinforcement Learning,” NeurIPS, 2025.
Used for: reinforcement learning as an alternative to conventional Text-to-SQL SFT and task-specific verifiable reward design.