Skip to content

fix: include actual values in truncated batch_max_tokens assert messages#1381

Open
Sarah-Salah wants to merge 1 commit into
ModelTC:mainfrom
Sarah-Salah:fix/truncated-assert-messages-api-start
Open

fix: include actual values in truncated batch_max_tokens assert messages#1381
Sarah-Salah wants to merge 1 commit into
ModelTC:mainfrom
Sarah-Salah:fix/truncated-assert-messages-api-start

Conversation

@Sarah-Salah

Copy link
Copy Markdown

What this PR does

Two assert statements in lightllm/server/api_start.py were meant to include
the offending values in their error message, but the f"but got ..." part was
written on a separate line, outside the assert. This PR folds those strings into
the assert messages so the values are actually shown.

Why

Because the f"but got {...}" line sits outside the assert, Python evaluates
it as a standalone expression statement and discards it. As a result, when a user
misconfigures batch_max_tokens, the AssertionError only shows the generic
message and silently drops the actual values, exactly the information needed to
debug the misconfiguration. This affects both the normal-mode and chunked-prefill
assertions.

Fix

Wrap each assert message in parentheses so the two string literals are combined
via implicit concatenation, keeping the offending values in the message.

Testing

  • No control-flow change, only the assertion message contents are affected.
  • Verified via AST that there are no more orphaned string expression statements
    in the file (the two stray f"but got ..." statements are gone).
  • python -m py_compile lightllm/server/api_start.py passes.
  • pre-commit (black + flake8) passes.

Signed-off-by: Sarah-Salah <11881117+Sarah-Salah@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request corrects the formatting of multi-line assertion messages in lightllm/server/api_start.py so that the full error message is properly included. The reviewer recommends replacing these assert statements with explicit if conditions that raise a ValueError, ensuring that these configuration checks are not bypassed when Python is executed with optimization flags.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +281 to +284
assert args.batch_max_tokens >= args.max_req_total_len, (
f"batch_max_tokens must >= max_req_total_len, "
f"but got {args.batch_max_tokens}, {args.max_req_total_len}"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using assert statements for validating production configuration or user-supplied arguments is risky because assertions can be globally disabled in Python when run with optimization flags (e.g., python -O). If disabled, these checks are completely bypassed, which can lead to unexpected runtime errors or silent failures later in the execution. Replacing them with explicit if checks that raise ValueError ensures that these critical constraints are always enforced.

Suggested change
assert args.batch_max_tokens >= args.max_req_total_len, (
f"batch_max_tokens must >= max_req_total_len, "
f"but got {args.batch_max_tokens}, {args.max_req_total_len}"
)
if args.batch_max_tokens < args.max_req_total_len:
raise ValueError(
f"batch_max_tokens must be >= max_req_total_len, "
f"but got {args.batch_max_tokens}, {args.max_req_total_len}"
)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review!

You're right that if ... raise ValueError is the textbook approach for validating
user-supplied config, since asserts are stripped under python -O.

That said, I'd like to keep this PR scoped to its original intent, restoring the
truncated assert messages, and keep the assert style here, for two reasons:

  • Consistency: this file uses assert as its standard pattern for startup argument
    checks (27 asserts vs 8 raise ValueError in api_start.py), including several in
    this same block. Converting just these two would leave the file mixing two idioms
    for the same kind of check.
  • The -O hardening would be incomplete anyway: the surrounding startup validations
    are all asserts too, so converting only these two wouldn't meaningfully protect
    against -O while the neighbours remain.

If the maintainers would prefer to migrate startup validation to explicit raises,
I'd be happy to open a separate issue/PR to do that consistently across the file.
Keeping this one focused on the message fix.

Comment on lines +291 to +294
assert args.batch_max_tokens >= args.chunked_prefill_size, (
"chunked prefill mode, batch_max_tokens must >= chunked_prefill_size, "
f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using assert statements for validating production configuration or user-supplied arguments is risky because assertions can be globally disabled in Python when run with optimization flags (e.g., python -O). If disabled, these checks are completely bypassed, which can lead to unexpected runtime errors or silent failures later in the execution. Replacing them with explicit if checks that raise ValueError ensures that these critical constraints are always enforced.

Suggested change
assert args.batch_max_tokens >= args.chunked_prefill_size, (
"chunked prefill mode, batch_max_tokens must >= chunked_prefill_size, "
f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}"
)
if args.batch_max_tokens < args.chunked_prefill_size:
raise ValueError(
"chunked prefill mode, batch_max_tokens must be >= chunked_prefill_size, "
f"but got {args.batch_max_tokens}, {args.chunked_prefill_size}"
)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review!

You're right that if ... raise ValueError is the textbook approach for validating
user-supplied config, since asserts are stripped under python -O.

That said, I'd like to keep this PR scoped to its original intent, restoring the
truncated assert messages, and keep the assert style here, for two reasons:

  • Consistency: this file uses assert as its standard pattern for startup argument
    checks (27 asserts vs 8 raise ValueError in api_start.py), including several in
    this same block. Converting just these two would leave the file mixing two idioms
    for the same kind of check.
  • The -O hardening would be incomplete anyway: the surrounding startup validations
    are all asserts too, so converting only these two wouldn't meaningfully protect
    against -O while the neighbours remain.

If the maintainers would prefer to migrate startup validation to explicit raises,
I'd be happy to open a separate issue/PR to do that consistently across the file.
Keeping this one focused on the message fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant