Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 129 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

## [0.3.11] - 2026-08-14

### Added

- `RemoteModelWrapper` (`textattack.models.wrappers.RemoteModelWrapper`): query a model served behind a remote HTTP API instead of running it locally. Request/response handling is adaptable to different endpoint schemas via `request_fn`/`response_fn`.
- New attack recipe `bad-characters` (`BadCharacters2021`, see
`textattack/attack_recipes/bad_characters_2021.py`) implementing
[Bad Characters: Imperceptible NLP Attacks](https://arxiv.org/abs/2106.09898):
perturbations invisible on some rendering systems (invisible characters,
homoglyphs, reorderings, deletions of zero-width characters), combining a
new `differential-evolution` search method (`WordSwapDifferentialEvolution`
transformation family) with new `LogitSum`/`NamedEntityRecognition`/
`TargetedStrict`/`TargetedBonus` goal functions ([#817](https://github.com/QData/TextAttack/issues/817)).
- New attack recipe `leap` (`LEAP2023`, see `textattack/attack_recipes/leap_2023.py`)
implementing LEAP: Efficient and Automated Test Method for NLP Software
([arXiv:2308.11284](https://arxiv.org/abs/2308.11284)). LEAP is a
Expand All @@ -28,9 +38,28 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
LEAP using ~6% fewer queries and running ~2.25x faster; under a 2000-query
budget (n=100), success collapses to ~23-24% for both, with LEAP still
modestly ahead on every metric but by a much smaller margin.
- `train` command: `--dataset-from-file` support, matching what `attack`/`eval`
already had. Point it at a Python module exposing `train_dataset`/
`eval_dataset` (each a `textattack.datasets.Dataset`), optionally
`path.py^prefix` for `prefix_train_dataset`/`prefix_eval_dataset` ([#625](https://github.com/QData/TextAttack/issues/625)).
- `HuggingFaceModelWrapper`: an optional `max_length` constructor argument,
forwarded to `.generate()` for raw `transformers` encoder-decoder
generation models. Left unset by default so a checkpoint's own
`generation_config` isn't overridden.
- Docs: the single-example `Attack.attack(text, label)` API (already existed,
wasn't documented) ([#673](https://github.com/QData/TextAttack/issues/673)); a working example attacking a raw
`transformers.BartForConditionalGeneration`/`T5ForConditionalGeneration`
via `Seq2SickCheng2018BlackBox` ([#772](https://github.com/QData/TextAttack/issues/772)); a "Multi-lingual attacks" section
listing the French/Spanish/Chinese recipes ([#423](https://github.com/QData/TextAttack/issues/423)).

### Changed

- Modernized CI: bumped GitHub Actions (`actions/checkout` v2->v4,
`actions/setup-python` v2->v5, `github/codeql-action/*` v1->v3), fixed
the pre-existing lint errors this surfaced, and re-enabled `pytest`
execution in CI (it had been disabled) ([#822](https://github.com/QData/TextAttack/issues/822)).
- `ChineseWordSwapHowNet`: cache HowNet replacement-word lookups to speed
up repeated candidate generation, without changing results ([#786](https://github.com/QData/TextAttack/issues/786)).
- Refactored `ParticleSwarmOptimization.perform_search` (shared base class)
to expose its per-iteration deltas as overridable hook methods
(`_initialize_velocities`, `_pre_iteration_setup`, `_compute_omega`,
Expand All @@ -54,6 +83,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

### Fixed

- `EDA` augmentation recipe (`textattack augment` CLI with `--recipe eda`):
didn't accept the default arguments defined on the `Augmentation`
superclass, since it didn't forward `**kwargs` to its component
`Augmenter` objects.
- `HuggingFaceDataset`: `shuffle()` (and `shuffle=True` at construction)
had no effect, since `datasets.Dataset.shuffle()` returns a new
`Dataset` rather than shuffling in place, and the return value wasn't
being assigned back ([#791](https://github.com/QData/TextAttack/issues/791)).
- `WordSwapChangeNumber._alter_number`: raised `ValueError: high is out
of bounds for int64` on negative numbers, since `int(num *
self.max_change) + 1` can compute a negative `change` that flips the
`randrange`/`randint` bounds ([#741](https://github.com/QData/TextAttack/issues/741)).
- `sentence_encoder.py`'s `get_angular_sim`: clamp `cos_sim` to `[-1, 1]`
before `torch.acos`, since floating-point error can push an equal pair
of embeddings' cosine similarity slightly above `1` (e.g. `1.00004`),
making `acos` return `NaN` instead of `1`.
- `CompositeTransformation`: iterated its sub-transformations' results
through a `set`, making output order (and therefore downstream
behavior relying on it) non-deterministic across runs; switched to a
list-based dedup that preserves order.
- Typo fix in `composite_transformation.py`'s docstring ("optoins" ->
"options").

Several correctness issues found while porting LEAP against its authors'
reference implementation, some of which also affect the pre-existing `pso`
recipe since `ParticleSwarmOptimizationLEAP` shares code with
Expand Down Expand Up @@ -88,3 +140,80 @@ recipe since `ParticleSwarmOptimizationLEAP` shares code with
unbounded `while True` loop with no fallback.
- Fixed `docs/api/search_methods.rst`'s `ParticleSwarmOptimizationLEAP`
section heading underline, which was shorter than the title.

A round of fixes for long-standing issues found during an issue-triage pass:

- `BERTAttackLi2020`: default `max_candidates` `48` -> `8`, since `48`
makes the masked-LM candidate search explode combinatorially on
multi-subword tokens, causing multi-hour runtimes ([#586](https://github.com/QData/TextAttack/issues/586)).
- `AttackedText.generate_new_attacked_text`: fixed corruption of the
`<SPLIT>` join token when a replaced word (e.g. "I") is itself a
character substring of `"<SPLIT>"` and directly follows it in the
joined text ([#631](https://github.com/QData/TextAttack/issues/631)).
- `WordSwapInflections`: restored matching against flair's current
`"upos-fast"` tags (`NOUN`/`VERB`/`ADJ`/`PROPN`/`AUX`), which
`AttackedText.pos_of_word_index` had switched to emitting directly a
while back, leaving the transformation silently returning zero
candidates for ordinary words ([#713](https://github.com/QData/TextAttack/issues/713), [#727](https://github.com/QData/TextAttack/issues/727)).
- `textattack.shared.utils.flair_tag`: cached one tagger per `tag_type`
instead of a single global slot, which silently reused whichever
tagger (POS or NER) loaded first for every later call regardless of
the requested `tag_type` -- corrupting POS/NER results for whichever
came second in the same process.
- `GreedyWordSwapWIR`: actually implemented `truncate_words_to` (a2t's
recipe had been passing it since an earlier PR, but the constructor
never accepted it, crashing on init) ([#754](https://github.com/QData/TextAttack/issues/754)); for `wir_method="gradient"`,
the truncation now also bounds the expensive `get_grad` call itself
(not just the cheap post-hoc index-scoring loop), and sorts
`indices_to_order` first since it can arrive in non-ascending order
from a `set`-derived source.
- `textattack/shared/validators.py`: the model-compatibility regex for
classification models only matched the pre-4.x
`transformers.modeling_<model>` layout; now also matches
`transformers.models.<model>.modeling_<model>` ([#722](https://github.com/QData/TextAttack/issues/722)). Also added a
matching entry for raw `transformers` encoder-decoder generation
classes (`T5ForConditionalGeneration`, `BartForConditionalGeneration`,
...), not just TextAttack's own `T5ForTextToText` helper, which used
to print a spurious compatibility warning for every attack against one
([#771](https://github.com/QData/TextAttack/issues/771)).
- `AttackArgs`: warn (rather than silently drop) when `num_examples` is
explicitly set alongside `num_successful_examples`, since the latter
overrides the former and users combining both had no way to tell why
`num_examples` came back `None` ([#728](https://github.com/QData/TextAttack/issues/728)).
- `AttackedText.words_diff_ratio`: comparing two Python lists with `!=`
yields a single bool, not an elementwise mask, so this always returned
`0` or `1` regardless of how many words actually differed; fixed by
comparing as numpy arrays ([#787](https://github.com/QData/TextAttack/issues/787)).
- `Augmenter.augment`: bound-retry the outer sampling loop so a
transformation with limited output diversity for a given input (e.g.
`BackTranslationAugmenter`'s random language chaining colliding on
short sentences) doesn't silently return fewer than
`transformations_per_example` unique augmentations ([#800](https://github.com/QData/TextAttack/issues/800)); a later
version of that same fix made `high_yield=True` mode plateau at roughly
half its previous output instead of scaling with
`transformations_per_example` (a single outer pass can add several
results at once in that mode), and its final downsampling step called
`random.sample()` on a `set`, which Python 3.11+ no longer accepts.
- `words_from_text`: strip allowed marks (quotes/hyphens/etc.) from both
ends of a word, not just the leading end, so a quoted word like
`"'CCC'"` doesn't keep its trailing quote ([#723](https://github.com/QData/TextAttack/issues/723)).
- `HuggingFaceModelWrapper`: route encoder-decoder generation models
(e.g. a raw `BartForConditionalGeneration` loaded directly from
`transformers`) through `.generate()` + decode instead of a plain
forward pass, which only returns logits and breaks text-to-text goal
functions expecting strings ([#771](https://github.com/QData/TextAttack/issues/771)). Routing prefers `model.can_generate()`
over `hasattr(model, "generate")`, since on `transformers` versions
predating `can_generate()`, every `PreTrainedModel` exposed `.generate`
regardless of whether it had a generation-capable head, risking
misrouting a seq2seq-backbone classification model.
- `Attack.cuda_`/`cpu_`: skip re-placing a `transformers.PreTrainedModel`
that already has an `hf_device_map` (i.e. was loaded with
`device_map=...` across multiple GPUs via `accelerate`), since forcing
it onto a single device breaks that placement ([#798](https://github.com/QData/TextAttack/issues/798)); `cpu_` was missing
this guard entirely, and the guard is now scoped to
`transformers.PreTrainedModel` specifically rather than any
`torch.nn.Module`, since this visitor also traverses non-HuggingFace
models reachable from a `Constraint`/`GoalFunction`/`Transformation`.
- `Trainer.training_step`/`evaluate_step`: pad to the longest sequence in
the batch (`padding=True`) instead of the tokenizer's static max
length, avoiding wasted compute on shorter batches ([#737](https://github.com/QData/TextAttack/issues/737)).
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
author = "UVA QData Lab"

# The full version, including alpha/beta/rc tags
release = "0.3.10"
release = "0.3.11"

# Set master doc to `index.rst`.
master_doc = "index"
Expand Down
Loading