fix: include actual values in truncated batch_max_tokens assert messages#1381
fix: include actual values in truncated batch_max_tokens assert messages#1381Sarah-Salah wants to merge 1 commit into
Conversation
Signed-off-by: Sarah-Salah <11881117+Sarah-Salah@users.noreply.github.com>
There was a problem hiding this comment.
Code Review
This pull request corrects the formatting of multi-line assertion messages in lightllm/server/api_start.py so that the full error message is properly included. The reviewer recommends replacing these assert statements with explicit if conditions that raise a ValueError, ensuring that these configuration checks are not bypassed when Python is executed with optimization flags.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| assert args.batch_max_tokens >= args.max_req_total_len, ( | ||
| f"batch_max_tokens must >= max_req_total_len, " | ||
| f"but got {args.batch_max_tokens}, {args.max_req_total_len}" | ||
| ) |
There was a problem hiding this comment.
Using assert statements for validating production configuration or user-supplied arguments is risky because assertions can be globally disabled in Python when run with optimization flags (e.g., python -O). If disabled, these checks are completely bypassed, which can lead to unexpected runtime errors or silent failures later in the execution. Replacing them with explicit if checks that raise ValueError ensures that these critical constraints are always enforced.
| assert args.batch_max_tokens >= args.max_req_total_len, ( | |
| f"batch_max_tokens must >= max_req_total_len, " | |
| f"but got {args.batch_max_tokens}, {args.max_req_total_len}" | |
| ) | |
| if args.batch_max_tokens < args.max_req_total_len: | |
| raise ValueError( | |
| f"batch_max_tokens must be >= max_req_total_len, " | |
| f"but got {args.batch_max_tokens}, {args.max_req_total_len}" | |
| ) |
There was a problem hiding this comment.
Thanks for the review!
You're right that if ... raise ValueError is the textbook approach for validating
user-supplied config, since asserts are stripped under python -O.
That said, I'd like to keep this PR scoped to its original intent, restoring the
truncated assert messages, and keep the assert style here, for two reasons:
- Consistency: this file uses
assertas its standard pattern for startup argument
checks (27 asserts vs 8raise ValueErrorinapi_start.py), including several in
this same block. Converting just these two would leave the file mixing two idioms
for the same kind of check. - The
-Ohardening would be incomplete anyway: the surrounding startup validations
are all asserts too, so converting only these two wouldn't meaningfully protect
against-Owhile the neighbours remain.
If the maintainers would prefer to migrate startup validation to explicit raises,
I'd be happy to open a separate issue/PR to do that consistently across the file.
Keeping this one focused on the message fix.
| assert args.batch_max_tokens >= args.chunked_prefill_size, ( | ||
| "chunked prefill mode, batch_max_tokens must >= chunked_prefill_size, " | ||
| f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}" | ||
| ) |
There was a problem hiding this comment.
Using assert statements for validating production configuration or user-supplied arguments is risky because assertions can be globally disabled in Python when run with optimization flags (e.g., python -O). If disabled, these checks are completely bypassed, which can lead to unexpected runtime errors or silent failures later in the execution. Replacing them with explicit if checks that raise ValueError ensures that these critical constraints are always enforced.
| assert args.batch_max_tokens >= args.chunked_prefill_size, ( | |
| "chunked prefill mode, batch_max_tokens must >= chunked_prefill_size, " | |
| f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}" | |
| ) | |
| if args.batch_max_tokens < args.chunked_prefill_size: | |
| raise ValueError( | |
| "chunked prefill mode, batch_max_tokens must be >= chunked_prefill_size, " | |
| f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}" | |
| ) |
There was a problem hiding this comment.
Thanks for the review!
You're right that if ... raise ValueError is the textbook approach for validating
user-supplied config, since asserts are stripped under python -O.
That said, I'd like to keep this PR scoped to its original intent, restoring the
truncated assert messages, and keep the assert style here, for two reasons:
- Consistency: this file uses
assertas its standard pattern for startup argument
checks (27 asserts vs 8raise ValueErrorinapi_start.py), including several in
this same block. Converting just these two would leave the file mixing two idioms
for the same kind of check. - The
-Ohardening would be incomplete anyway: the surrounding startup validations
are all asserts too, so converting only these two wouldn't meaningfully protect
against-Owhile the neighbours remain.
If the maintainers would prefer to migrate startup validation to explicit raises,
I'd be happy to open a separate issue/PR to do that consistently across the file.
Keeping this one focused on the message fix.
What this PR does
Two
assertstatements inlightllm/server/api_start.pywere meant to includethe offending values in their error message, but the
f"but got ..."part waswritten on a separate line, outside the assert. This PR folds those strings into
the assert messages so the values are actually shown.
Why
Because the
f"but got {...}"line sits outside theassert, Python evaluatesit as a standalone expression statement and discards it. As a result, when a user
misconfigures
batch_max_tokens, theAssertionErroronly shows the genericmessage and silently drops the actual values, exactly the information needed to
debug the misconfiguration. This affects both the normal-mode and chunked-prefill
assertions.
Fix
Wrap each assert message in parentheses so the two string literals are combined
via implicit concatenation, keeping the offending values in the message.
Testing
in the file (the two stray
f"but got ..."statements are gone).python -m py_compile lightllm/server/api_start.pypasses.pre-commit(black + flake8) passes.