diff --git a/CHANGELOG.md b/CHANGELOG.md index fd26aa42..f0a09328 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,9 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [Unreleased] +## [0.3.11] - 2026-08-14 + ### Added - `RemoteModelWrapper` (`textattack.models.wrappers.RemoteModelWrapper`): query a model served behind a remote HTTP API instead of running it locally. Request/response handling is adaptable to different endpoint schemas via `request_fn`/`response_fn`. +- New attack recipe `bad-characters` (`BadCharacters2021`, see + `textattack/attack_recipes/bad_characters_2021.py`) implementing + [Bad Characters: Imperceptible NLP Attacks](https://arxiv.org/abs/2106.09898): + perturbations invisible on some rendering systems (invisible characters, + homoglyphs, reorderings, deletions of zero-width characters), combining a + new `differential-evolution` search method (`WordSwapDifferentialEvolution` + transformation family) with new `LogitSum`/`NamedEntityRecognition`/ + `TargetedStrict`/`TargetedBonus` goal functions ([#817](https://github.com/QData/TextAttack/issues/817)). - New attack recipe `leap` (`LEAP2023`, see `textattack/attack_recipes/leap_2023.py`) implementing LEAP: Efficient and Automated Test Method for NLP Software ([arXiv:2308.11284](https://arxiv.org/abs/2308.11284)). LEAP is a @@ -28,9 +38,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). LEAP using ~6% fewer queries and running ~2.25x faster; under a 2000-query budget (n=100), success collapses to ~23-24% for both, with LEAP still modestly ahead on every metric but by a much smaller margin. +- `train` command: `--dataset-from-file` support, matching what `attack`/`eval` + already had. Point it at a Python module exposing `train_dataset`/ + `eval_dataset` (each a `textattack.datasets.Dataset`), optionally + `path.py^prefix` for `prefix_train_dataset`/`prefix_eval_dataset` ([#625](https://github.com/QData/TextAttack/issues/625)). +- `HuggingFaceModelWrapper`: an optional `max_length` constructor argument, + forwarded to `.generate()` for raw `transformers` encoder-decoder + generation models. Left unset by default so a checkpoint's own + `generation_config` isn't overridden. +- Docs: the single-example `Attack.attack(text, label)` API (already existed, + wasn't documented) ([#673](https://github.com/QData/TextAttack/issues/673)); a working example attacking a raw + `transformers.BartForConditionalGeneration`/`T5ForConditionalGeneration` + via `Seq2SickCheng2018BlackBox` ([#772](https://github.com/QData/TextAttack/issues/772)); a "Multi-lingual attacks" section + listing the French/Spanish/Chinese recipes ([#423](https://github.com/QData/TextAttack/issues/423)). ### Changed +- Modernized CI: bumped GitHub Actions (`actions/checkout` v2->v4, + `actions/setup-python` v2->v5, `github/codeql-action/*` v1->v3), fixed + the pre-existing lint errors this surfaced, and re-enabled `pytest` + execution in CI (it had been disabled) ([#822](https://github.com/QData/TextAttack/issues/822)). +- `ChineseWordSwapHowNet`: cache HowNet replacement-word lookups to speed + up repeated candidate generation, without changing results ([#786](https://github.com/QData/TextAttack/issues/786)). - Refactored `ParticleSwarmOptimization.perform_search` (shared base class) to expose its per-iteration deltas as overridable hook methods (`_initialize_velocities`, `_pre_iteration_setup`, `_compute_omega`, @@ -54,6 +83,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ### Fixed +- `EDA` augmentation recipe (`textattack augment` CLI with `--recipe eda`): + didn't accept the default arguments defined on the `Augmentation` + superclass, since it didn't forward `**kwargs` to its component + `Augmenter` objects. +- `HuggingFaceDataset`: `shuffle()` (and `shuffle=True` at construction) + had no effect, since `datasets.Dataset.shuffle()` returns a new + `Dataset` rather than shuffling in place, and the return value wasn't + being assigned back ([#791](https://github.com/QData/TextAttack/issues/791)). +- `WordSwapChangeNumber._alter_number`: raised `ValueError: high is out + of bounds for int64` on negative numbers, since `int(num * + self.max_change) + 1` can compute a negative `change` that flips the + `randrange`/`randint` bounds ([#741](https://github.com/QData/TextAttack/issues/741)). +- `sentence_encoder.py`'s `get_angular_sim`: clamp `cos_sim` to `[-1, 1]` + before `torch.acos`, since floating-point error can push an equal pair + of embeddings' cosine similarity slightly above `1` (e.g. `1.00004`), + making `acos` return `NaN` instead of `1`. +- `CompositeTransformation`: iterated its sub-transformations' results + through a `set`, making output order (and therefore downstream + behavior relying on it) non-deterministic across runs; switched to a + list-based dedup that preserves order. +- Typo fix in `composite_transformation.py`'s docstring ("optoins" -> + "options"). + Several correctness issues found while porting LEAP against its authors' reference implementation, some of which also affect the pre-existing `pso` recipe since `ParticleSwarmOptimizationLEAP` shares code with @@ -88,3 +140,80 @@ recipe since `ParticleSwarmOptimizationLEAP` shares code with unbounded `while True` loop with no fallback. - Fixed `docs/api/search_methods.rst`'s `ParticleSwarmOptimizationLEAP` section heading underline, which was shorter than the title. + +A round of fixes for long-standing issues found during an issue-triage pass: + +- `BERTAttackLi2020`: default `max_candidates` `48` -> `8`, since `48` + makes the masked-LM candidate search explode combinatorially on + multi-subword tokens, causing multi-hour runtimes ([#586](https://github.com/QData/TextAttack/issues/586)). +- `AttackedText.generate_new_attacked_text`: fixed corruption of the + `` join token when a replaced word (e.g. "I") is itself a + character substring of `""` and directly follows it in the + joined text ([#631](https://github.com/QData/TextAttack/issues/631)). +- `WordSwapInflections`: restored matching against flair's current + `"upos-fast"` tags (`NOUN`/`VERB`/`ADJ`/`PROPN`/`AUX`), which + `AttackedText.pos_of_word_index` had switched to emitting directly a + while back, leaving the transformation silently returning zero + candidates for ordinary words ([#713](https://github.com/QData/TextAttack/issues/713), [#727](https://github.com/QData/TextAttack/issues/727)). +- `textattack.shared.utils.flair_tag`: cached one tagger per `tag_type` + instead of a single global slot, which silently reused whichever + tagger (POS or NER) loaded first for every later call regardless of + the requested `tag_type` -- corrupting POS/NER results for whichever + came second in the same process. +- `GreedyWordSwapWIR`: actually implemented `truncate_words_to` (a2t's + recipe had been passing it since an earlier PR, but the constructor + never accepted it, crashing on init) ([#754](https://github.com/QData/TextAttack/issues/754)); for `wir_method="gradient"`, + the truncation now also bounds the expensive `get_grad` call itself + (not just the cheap post-hoc index-scoring loop), and sorts + `indices_to_order` first since it can arrive in non-ascending order + from a `set`-derived source. +- `textattack/shared/validators.py`: the model-compatibility regex for + classification models only matched the pre-4.x + `transformers.modeling_` layout; now also matches + `transformers.models..modeling_` ([#722](https://github.com/QData/TextAttack/issues/722)). Also added a + matching entry for raw `transformers` encoder-decoder generation + classes (`T5ForConditionalGeneration`, `BartForConditionalGeneration`, + ...), not just TextAttack's own `T5ForTextToText` helper, which used + to print a spurious compatibility warning for every attack against one + ([#771](https://github.com/QData/TextAttack/issues/771)). +- `AttackArgs`: warn (rather than silently drop) when `num_examples` is + explicitly set alongside `num_successful_examples`, since the latter + overrides the former and users combining both had no way to tell why + `num_examples` came back `None` ([#728](https://github.com/QData/TextAttack/issues/728)). +- `AttackedText.words_diff_ratio`: comparing two Python lists with `!=` + yields a single bool, not an elementwise mask, so this always returned + `0` or `1` regardless of how many words actually differed; fixed by + comparing as numpy arrays ([#787](https://github.com/QData/TextAttack/issues/787)). +- `Augmenter.augment`: bound-retry the outer sampling loop so a + transformation with limited output diversity for a given input (e.g. + `BackTranslationAugmenter`'s random language chaining colliding on + short sentences) doesn't silently return fewer than + `transformations_per_example` unique augmentations ([#800](https://github.com/QData/TextAttack/issues/800)); a later + version of that same fix made `high_yield=True` mode plateau at roughly + half its previous output instead of scaling with + `transformations_per_example` (a single outer pass can add several + results at once in that mode), and its final downsampling step called + `random.sample()` on a `set`, which Python 3.11+ no longer accepts. +- `words_from_text`: strip allowed marks (quotes/hyphens/etc.) from both + ends of a word, not just the leading end, so a quoted word like + `"'CCC'"` doesn't keep its trailing quote ([#723](https://github.com/QData/TextAttack/issues/723)). +- `HuggingFaceModelWrapper`: route encoder-decoder generation models + (e.g. a raw `BartForConditionalGeneration` loaded directly from + `transformers`) through `.generate()` + decode instead of a plain + forward pass, which only returns logits and breaks text-to-text goal + functions expecting strings ([#771](https://github.com/QData/TextAttack/issues/771)). Routing prefers `model.can_generate()` + over `hasattr(model, "generate")`, since on `transformers` versions + predating `can_generate()`, every `PreTrainedModel` exposed `.generate` + regardless of whether it had a generation-capable head, risking + misrouting a seq2seq-backbone classification model. +- `Attack.cuda_`/`cpu_`: skip re-placing a `transformers.PreTrainedModel` + that already has an `hf_device_map` (i.e. was loaded with + `device_map=...` across multiple GPUs via `accelerate`), since forcing + it onto a single device breaks that placement ([#798](https://github.com/QData/TextAttack/issues/798)); `cpu_` was missing + this guard entirely, and the guard is now scoped to + `transformers.PreTrainedModel` specifically rather than any + `torch.nn.Module`, since this visitor also traverses non-HuggingFace + models reachable from a `Constraint`/`GoalFunction`/`Transformation`. +- `Trainer.training_step`/`evaluate_step`: pad to the longest sequence in + the batch (`padding=True`) instead of the tokenizer's static max + length, avoiding wasted compute on shorter batches ([#737](https://github.com/QData/TextAttack/issues/737)). diff --git a/docs/conf.py b/docs/conf.py index 1ead5da5..5fb1bcbf 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -21,7 +21,7 @@ author = "UVA QData Lab" # The full version, including alpha/beta/rc tags -release = "0.3.10" +release = "0.3.11" # Set master doc to `index.rst`. master_doc = "index"