It seems that Chinese input is not supported

[zhouruicn]

It seems that Chinese input is not supported. If you input Chinese, it returns incomprehensible content. The same model works normally for inference in other frameworks.

[zhouruicn]

[mukel]

The streaming decoding doesn't work correctly for some characters like emojis, which I guess also applies to multibyte characters.
Try passing --stream false, this will decode all the tokens at once.
You observe the bad behavior with --prompt "Print 5 emojis" (regular streaming prints some odd characters, while --stream false works correctly)

[zhouruicn]

You mentioned this phenomenon: if you don't use stream, the output is recognizable Chinese characters, but the displayed output indicates that the input content was not recognized. I suspect the issue might be due to the prompt not being correctly recognized during the encoding process.