[FIP-40] merge fluss-rust#3401
Closed
fresh-borzoni wants to merge 323 commits into
Closed
Conversation
…CompactedRow::from_bytes(), these fields can be skipped as current rust client implementation does not check schema and already passes row_type (apache#190)
Add 9 RPC message wrapper types: - alter_database, alter_table (DDL operations) - get_table_stats (table statistics) - list_database_summaries (database listing with summaries) - create_acls, list_acls, drop_acls (ACL management) - describe_cluster_configs, alter_cluster_configs (cluster configuration) Each wrapper follows the standard pattern: a request struct wrapping the proto-generated type, implementing RequestBody (tying to ApiKey and ResponseBody), WriteType, and ReadType. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…apache#611) * [c++] Add MAP/ROW types, unify complex types under Value handle * address review * add int family tests for nested cases
…ases in write path (apache#635) The write path (sender, accumulator, bucket_assigner) and RPC messages (produce_log, put_kv) used raw i64/i32 for table/partition/bucket IDs instead of the semantic type aliases defined in lib.rs. This makes the code inconsistent with newer modules that already use the aliases. Changes: - cluster.rs: get_table_id() returns Option<TableId>, get_table_id_by_path() returns &HashMap<TablePath, TableId>, get_bucket_locations() takes TableId - produce_log.rs, put_kv.rs: new() takes TableId - sender.rs: build_write_request/send_and_handle_response/handle_write_response take TableId; BucketResponse trait returns BucketId - bucket_assigner.rs: BucketAssigner trait uses BucketId throughout - accumulator.rs: test helper uses BucketId - remote_log.rs: test helper uses TableId/BucketId Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [rust] Add RPC message wrappers for extended operations Add message wrappers for the remaining 1.x RPC APIs: - KV snapshot lifecycle: acquire/release/drop lease, list, metadata, latest snapshots, lake snapshot - Server management: add/remove server tag, rebalance + progress + cancel, get cluster health, list remote log manifests - Producer offsets: register/get/delete - ScanKv (API 1061): full KV-table bucket scan request/response * Add GoalType and ServerTag domain enums for RPC wrappers Replace raw i32 with proper Rust enums matching the Java Fluss definitions: GoalType (ReplicaDistribution/LeaderDistribution/RackAware) and ServerTag (PermanentOffline/TemporaryOffline). Addresses reviewer feedback on PR apache#630. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply TableId/PartitionId/BucketId type aliases to RPC messages Use the i64/i32 type aliases from lib.rs instead of raw primitives in the new RPC message wrappers introduced by this PR. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [elixir] feat: Add get_table_info admin call * Include schema in table_info and add get_table_schema admin call
* [rust] Add Fluss 1.x protocol support to the admin client Add 27 new admin methods to FlussAdmin: - Database/table extensions: list_database_summaries, alter_database, alter_table, get_table_stats - KV snapshot operations: get_latest_kv_snapshots, get_kv_snapshot_metadata, create_kv_snapshot_lease, get_lake_snapshot - ACL management: create_acls, list_acls, drop_acls - Cluster configuration: describe_cluster_configs, alter_cluster_configs - Server management: add_server_tag, remove_server_tag, rebalance, list_rebalance_progress, cancel_rebalance - Producer offsets: register_producer_offsets, get_producer_offsets, delete_producer_offsets - Monitoring: get_cluster_health, list_remote_log_manifests - KV snapshots: list_kv_snapshots, release_kv_snapshot_lease, drop_kv_snapshot_lease * Address reviewer feedback: add domain enums, type aliases, expose readable - Add ClusterHealthStatus enum (Green/Yellow/Red/Unknown) to cluster_health.rs - Add RebalanceStatus enum (NotStarted..Timeout) to rebalance.rs - Replace raw i32/i64 with BucketId/TableId/PartitionId aliases in kv_snapshot.rs, lake_snapshot.rs, rebalance.rs, producer_offsets.rs - Expose readable parameter in admin.get_lake_snapshot() instead of hardcoding None Addresses reviewer feedback on PR apache#631. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply type aliases to remaining metadata types (table_stats, kv_snapshot_lease) Extends the BucketId/TableId/PartitionId alias consistency fix to table_stats.rs (BucketStatsRequest, BucketStats) and kv_snapshot_lease.rs (KvSnapshotLeaseForBucket, KvSnapshotLeaseForTable). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update admin.rs to use GoalType/ServerTag enums after rebase Now that pr/3 provides GoalType and ServerTag enums in the RPC wrappers, update the admin client methods to use them in their public signatures too. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Apply TableId/PartitionId/BucketId type aliases to admin methods Update the new admin method signatures introduced by this PR to use the i64/i32 type aliases from lib.rs instead of raw primitives, matching the underlying RPC message wrappers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * Address reviewer feedback round 2 - Add `RegisterProducerResult` enum mirroring Java's `RegisterResult` (Created=0, AlreadyExists=1); `register_producer_offsets` now returns `Option<RegisterProducerResult>` instead of raw `Option<i32>`. - Expose `comment: Option<&str>` on `alter_database` and the `AlterDatabaseRequest` wrapper (was hardcoded to `None`). - Replace fully-qualified paths with imports: `BucketStatsRequest` in admin.rs, `PbDatabaseSummary` in metadata/database.rs, and `AlterConfig` self-reference in metadata/table_change.rs. - Rename `create_acls(acl)` to `create_acls(acls)` and `drop_acls(acl_filter)` to `drop_acls(acl_filters)` for plural consistency with the `Vec` argument type. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: warmbupt <warmbupt@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
@fresh-borzoni , please rebase this pr. |
loserwang1024
approved these changes
Jun 29, 2026
loserwang1024
left a comment
loserwang1024
left a comment
Contributor
There was a problem hiding this comment.
LGTM, please rebase this PR
Contributor
|
Thanks for the pr. |
Contributor
|
I'll have a review in next few days |
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
c232100 to
af7dd10
Compare
Member
Author
|
@luoyuxia @loserwang1024 @beryllw I brought a new sub-tree here, PTAL |
beryllw
reviewed
Jul 3, 2026
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
af7dd10 to
b6c71c6
Compare
…pache#647) This commit brings the Elixir's create_table descriptor to full parity with the native builder by also supporting :bucket_keys, :partition_keys, :custom_properties & comment. The optional fields are bundled into a Fluss.TableDescriptor.Options struct decoded across the NIF boundary as a NifStruct (NifTableOptions), rather than growing table_descriptor_new to seven positional arguments. This mirrors the existing NifDatabaseDescriptor pattern and the Python binding, which builds the core descriptor with unconditional .properties / .custom_properties / .partitioned_by / .distributed_by calls. :properties is now a map (was a list of {k, v} tuples), fixing a write/read asymmetry — get_table_info already reports properties as a map — and matching :custom_properties.
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
66f94bc to
f81e98b
Compare
luoyuxia
approved these changes
Jul 4, 2026
luoyuxia
left a comment
luoyuxia
left a comment
Contributor
There was a problem hiding this comment.
@fresh-borzoni LGTM. Please feel free to merge it with keeping the commit history.
Btw, after that, you can send a email to dev channel to announce it.
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
f81e98b to
c577ee3
Compare
Member
Author
|
@luoyuxia Thanks! Will do, once CI/CD is green. |
Member
Author
|
pushed on main via filter-repo, all clean and good, closing PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #3407
Brings the Rust, Python, C++, Elixir clients (apache/fluss-rust) into this repo under fluss-rust/, per FIP-40 (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421957144)