Bug of utils/string.py/words_from_text()

[kaijiezhu11]

Hi,

In this function words_from_text() in utils/string.py (used in AttackText for generating words list of a given sentence),

It seems that the function has difficulty handling quoted words.

For example, if an input is " AAA BBB 'CCC' ", the function words_from_text() returns AAA BBB and CCC', the left quote is deleted but the right quote is still reserved.

I would appreciate it if you could kindly take a look at this and see if there is a way to resolve the issue.

[qiyanjun]

Confirmed — the code only stripped leading quote/hyphen/etc. characters (lstrip), not trailing ones, so "'CCC'" kept its closing quote. Opened #831 which switches both strip calls to strip() (both ends); verified this doesn't affect contractions or hyphenated compounds since those only have the character in the middle of the word, plus added a regression test. Will close automatically once that merges.