feat: add support for alter schema drop vector index - #1991
Conversation
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
There was a problem hiding this comment.
Pull request overview
Adds client support for dropping a named vector’s index via the schema REST endpoint, mirroring the existing “delete property index” capability in the collections config API.
Changes:
- Added
delete_vector_index(vector_name)to the collection config executor, issuingDELETE /v1/schema/{className}/vectors/{vectorIndexName}/index. - Added sync and async type stubs for
delete_vector_index(...).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| weaviate/collections/config/executor.py | Implements the new delete-vector-index operation via the config executor. |
| weaviate/collections/config/sync.pyi | Exposes the new method in the sync config type stub. |
| weaviate/collections/config/async_.pyi | Exposes the new method in the async config type stub. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d728dd9 to
7e5cd7f
Compare
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Secrets | View in Orca |
3657487 to
736a7cd
Compare
| NONE: The index of this vector has been dropped, see ``collection.config.delete_vector_index``. | ||
| The vector data is still stored, but it cannot be searched. This value is reported by the | ||
| server only, it cannot be used to configure a vector. | ||
| """ |
There was a problem hiding this comment.
Leaving as-is for now: no public Configure.VectorIndex.* factory emits NONE (only the internal create models reference the field), and the server rejects "none" on both create and update, so the exposure is constructing private classes directly. Can add a validator in a follow-up if users actually hit it.
7e5cd7f to
4e2a580
Compare
736a7cd to
be6c8dc
Compare
4e2a580 to
2dbfaed
Compare
|
thought: Do you think this PR could do with fewer tests? The new endpoint is very simple and it seems to me like one "happy path" test will catch most things. Other tests verify Weaviate's behavior more than client's logic, and some may add up to 30s of pipeline time. |
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
@bevzzz fair point — slimmed in b92dbb7, keeping the coverage but moving it down the pyramid: the integration test is now a single happy-path journey (create → drop → poll → sibling index intact + searchable). The unknown-name case moved to the mock suite (422 → |
|
@copilot review |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1991 +/- ##
==========================================
+ Coverage 86.64% 88.55% +1.91%
==========================================
Files 300 304 +4
Lines 23172 23597 +425
==========================================
+ Hits 20077 20897 +820
+ Misses 3095 2700 -395 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
9d7b916 to
ccad4cc
Compare
Follow-up to the `delete_vector_index` support, addressing review findings. After a successful drop, Weaviate keeps the vector in the schema as `vectorIndexType: "none"` with no `vectorIndexConfig`. The client asserted that every named vector has an index config, so `collection.config.get()` and `client.collections.list_all()` raised `AssertionError` for every collection in the cluster once any vector index had been dropped. `_NamedVectorConfig. vector_index_config` is now optional, `VectorIndexType` gained a server-reported `NONE` member and `to_dict()` round-trips it. `collection.config.update()` on a dropped vector raised a bare `KeyError: 'vectorIndexConfig'` from the schema merge. Both the current and the deprecated merge paths now go through one helper that raises a `WeaviateInvalidInputError` explaining that a dropped index cannot be re-created. The docstring claimed the index could be regenerated and that a missing vector raises `WeaviateInvalidInputError`. Neither is true: Weaviate rejects re-creating a dropped index, and an unknown vector name comes back as a 422. It now also documents that the endpoint is experimental and needs `ENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=true`, that only named vectors can be dropped, and that the drop is applied asynchronously. The error message no longer blames a missing vector for what is usually a disabled endpoint. Tests: unit coverage for parsing, exporting and updating a dropped vector, mock coverage for the request path and the disabled-endpoint response, and integration coverage gated at 1.39.0. The CI compose file enables the experimental endpoint; that flag can be dropped once 1.39.0 is GA. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Replace the assert on dropped-vector schema shape with an explicit SchemaValidationError so a named vector missing vectorIndexConfig fails fast even under python -O; pinned by a new parser unit test - Slim the integration test to the happy path: the unknown-name error contract moved to the mock suite (422 -> UnexpectedStatusCodeError) and the redundant list_all/invalid-input assertions are covered by the existing unit and mock tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A single-vector (non-named) collection whose vector index is dropped with `collection.config.delete_vector_index` comes back from the server with no top-level `vectorizer` (and no `vectorConfig`, `vectorIndexType` or `vectorIndexConfig`). `__get_vectorizer` accessed `schema["vectorizer"]` unguarded and raised `KeyError: 'vectorizer'`, so both `config.get()` and `collections.list_all()` crashed on such a collection. Return `None` when the key is absent, matching how a dropped named vector yields `vector_index_config is None`. Add parser tests for both entry points. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ccad4cc to
16ea348
Compare
Verified against the Weaviate server source that these comments were wrong: - The endpoint is still experimental and off by default in current main; it is not enabled by default at 1.39.0 GA. The server rejects the drop unless ENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=true (usecases/schema/property.go:255). Fix the misleading ci/docker-compose.yml comment. - A legacy single-vector collection cannot reach the "no vectorConfig, no vectorizer" shape: the server rejects dropping its index because len(class.VectorConfig) == 0 (property.go:288), and setClassDefaults always forces a non-empty top-level vectorizer for legacy classes (class.go:750). That shape can only come from a named-vector collection whose vectors were all dropped. Reword the __get_vectorizer guard comment and rename the parser tests accordingly. Behavior and assertions are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
dirkkul
left a comment
There was a problem hiding this comment.
a few smaller things, none of them blocking:
-
the
delete_vector_indexdocstring goes stale as soon as the endpoint is promoted or the status code changes: it namesENABLE_EXPERIMENTAL_ALTER_SCHEMA_DROP_VECTOR_INDEX_ENDPOINT=trueand says the server answers 500 without it. someone reading it after that sets an env var the server no longer looks at. could we say the endpoint is experimental and may be disabled server-side, without naming the flag or the status code? -
in
test/collection/test_config_methods.py,test_collection_config_simple_from_json_with_dropped_vector_indexandtest_collection_config_simple_from_json_all_vectors_droppedparse the same schemas as the two non-simple tests above them -__get_vector_config(schema, simple)never readssimple, and__get_vectorizertakes no such parameter. the simple path does build a different dataclass, so could we keep one of the two as thelist_all()guard and drop the other? -
test_delete_vector_index_endpoint_disabledoverlaps the 422 case already insidetest_delete_vector_index- both assert that a non-OK status comes back asUnexpectedStatusCodeErrorcarrying the status code. the one thing it adds is that the server's error message reaches the exception, so could that move intotest_delete_vector_indexas another status/message case instead of standing up its own mock server?
reviewed with claude code, I went through every comment below myself
| update.merge_with_existing(schema) | ||
|
|
||
|
|
||
| def test_updating_vector_next_to_dropped_vector_index() -> None: |
There was a problem hiding this comment.
This seems like a test that should be in Weaviate and not the python client
There was a problem hiding this comment.
But it's basically testing a python functionality, the merge_with_existing. If this test would go as a go acceptance test it wouldn't invoke the same logic imho. It's not even using a Weaviate instance.
- config.py: __existing_vector_index_config raised a raw KeyError when the collection had no vectors left (server omits vectorConfig once every named vector is dropped). Guard the key so the intended WeaviateInvalidInputError is raised instead. Add a regression test. - config_methods.py: __get_vector_config reported "no vectorIndexConfig" for a vectorIndexType the client does not know, even though the config was present (an older client against a newer server). Branch on "vectorIndexConfig" in the named vector and give the unknown-type case its own message. Add a regression test. - executor.py: delete_vector_index no longer names the env flag or the 500 status in its docstring (both go stale when the endpoint is promoted); it now returns None instead of a bool that could only ever be True. Regenerate stubs. - tests: fold the disabled-endpoint case into test_delete_vector_index (its only unique check is that the server message reaches the exception) and drop the redundant simple-parser test; the all-vectors-dropped simple test remains the list_all() guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The round-4 commit changed delete_vector_index to return None but only updated the mock test and stubs; the integration assertion still expected True and would fail on every >=1.39 CI job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Summary
delete_vector_index(vector_name)method to collection config, allowing users to drop a named vector's index viaDELETE /v1/schema/{className}/vectors/{vectorIndexName}/indexdelete_property_indexmethodCloses #1990