fix: Add the ability to skip existing points

[robcaulk]

Migrating or syncing large active collections will inherently miss some points that are upserted during the migration (if a UUID is upserted behind the migration progress). Therefore, a second migration pass can clean these up. The proposed PR adds a a skip-existing flag that, when activated, will sync any missed points but avoid re-upserting al lthe existing points in the database.

As a secondary feature, this can be a component of a syncing mechanism that keeps various databases synced between regions.

The code changes for this PR were generated using AI

Claude's description/doc:

--migration.skip-existing
Add this flag when running a Qdrant-to-Qdrant migration against a target that already has some (or all) of the data:


docker run --net=host --rm -it registry.cloud.qdrant.io/library/qdrant-migration qdrant \
    --source.url 'http://source-hostname:6334' \
    --source.collection 'source-collection' \
    --target.url 'https://example.qdrant.io:6334' \
    --target.api-key 'qdrant-key' \
    --target.collection 'target-collection' \
    --migration.skip-existing
What it does: Before each batch upsert, fetches the batch's point IDs from the target (no vectors/payload — lightweight). Any IDs already present are filtered out; only new points are upserted.

When to use it:

Re-running a migration that previously completed, where you want to avoid overwriting target data
Target collection was pre-populated from another source and you want additive-only behavior
When not to use it:

Fresh migrations (nothing in the target) — adds one extra read RPC per batch with no benefit
You want the source to overwrite stale target data (default upsert behavior)
Cost: One extra read RPC to the target per batch. With the default batch size of 50, this doubles the number of target RPCs.

[Anush008]

Hi @robcaulk.
Thanks for taking the time to contribute. I've pushed a small improvment.

[Anush008]

Could you please re-commit with a verified signature? That's a requirement for the merge.

If you use 1Password, you can refer to https://1password.com/blog/git-commit-signing.

[Anush008]

Bumping this @robcaulk