fix: allow scalar subquery as LIMIT row count (#2359) - #2464
Merged
manticore-projects merged 1 commit intoAug 14, 2026
Merged
Conversation
PostgreSQL allows any expression, including a scalar subquery, as the LIMIT row count, e.g. LIMIT (SELECT COUNT(*) FROM t WHERE ...) or LIMIT GREATEST(0, (SELECT ...)). JSQLParser rejected these. PlainSelect disambiguated the ClickHouse "LIMIT ... BY ..." branch from a plain limit with a numeric LOOKAHEAD(7) on LimitBy(). A numeric lookahead cannot see past a long parenthesized subquery, so for LIMIT (subquery) it wrongly committed to the LIMIT BY branch and then failed at the missing BY keyword. (Short subqueries such as LIMIT (SELECT 1) happened to stay under the lookahead window and worked, which is why the bug only surfaced for longer ones.) The disambiguation is moved to where it belongs: parse the LIMIT row count once via LimitWithOffset() (which already accepts a parenthesized subquery), then check the immediately following token for BY. A token scan for BY would not work, because a subquery may contain ORDER BY (a K_BY). The now-unused LimitBy() production is removed; LimitWithOffset already carries byExpressions, so no AST or public API change. All LIMIT shapes keep working: LIMIT n, LIMIT n, m, LIMIT n OFFSET m, OFFSET m LIMIT n, LIMIT ALL, and ClickHouse LIMIT n BY ... / LIMIT n, m BY ... Fixes JSQLParser#2359 Signed-off-by: 付典 <fudianchn@gmail.com>
fudianchn
marked this pull request as draft
August 13, 2026 18:28
fudianchn
marked this pull request as ready for review
August 13, 2026 19:23
Contributor
|
This is the much better solution of course, thank you for fixing this! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Allow any expression, including a scalar subquery, as the LIMIT row count, e.g.:
JSQLParser currently rejects these with
Encountered: <EOF> ... Was expecting: "BY".Why / Root cause
As you noted in the issue, a
LOOKAHEADis too tight.PlainSelectdisambiguated the ClickHouseLIMIT ... BY ...branch from a plain limit with a numericLOOKAHEAD(7)onLimitBy(). A numeric lookahead cannot see past a long parenthesized subquery, so forLIMIT (SELECT COUNT(*) FROM ... WHERE ...)it wrongly committed to theLIMIT BYbranch and then failed at the missingBY. Short subqueries such asLIMIT (SELECT 1)happened to stay under the 7-token window and worked, which is why only longer ones failed.A naive fix (scanning ahead for a
BYtoken) would not work, because a subquery may legitimately containORDER BY, whoseBYis the sameK_BYtoken.How
The disambiguation is moved to where it belongs: parse the LIMIT row count once via
LimitWithOffset()(which already accepts a parenthesized subquery through itsLOOKAHEAD(3) ParenthesedSelect()branch), then check the immediately following token forBY:LimitWithOffsetalready carriesbyExpressions, so there is no AST and no public API change. The now-unusedLimitBy()production is removed (it was only ever called from this one site).Scope
All existing LIMIT shapes keep working:
LIMIT n,LIMIT n, m,LIMIT n OFFSET m,OFFSET m LIMIT n,LIMIT ALL, and ClickHouseLIMIT n BY .../LIMIT n, m BY .... The previously-unsupportedOFFSET m LIMIT n BY xordering stays unsupported (out of scope, unchanged).Testing
LimitExpressionTest.testIssue2359reproduces the issue (fails onmaster, passes with this change) and asserts the row count is aParenthesedSelectstored onlimit(notlimitBy).testLimitByClickHouseUnchangedguards the rewrittenLIMIT ... BY ...path.Local:
spotlessApply,checkstyleMain/Test,spotbugsMain,pmdMainclean;SelectTest(725),ClickHouseTest(15),LimitExpressionTest(6) and the full test suite green apart from pre-existing failures unrelated to this change (ParserKeywordsUtilsTestwriting a temp file toC:\WINDOWS, and MockitoMockMakerinit on JDK 21).Performance
gradle jmh,JSQLParserBenchmark.parseSQLStatementsonperformance.sql,version=latest, 10 forks × 10 iterations (100 samples) on a 32-core host:LOOKAHEAD(7) LimitBy())3.598 ± 0.0173.603 ± 0.017The
+0.14%delta lies within the 99.9% confidence intervals (the CIs overlap almost entirely): no regression.Verification of the original issue
fails on
masterand parses + round-trips with this change.Fixes #2359