Skip to content

[flink] Add describe buckets procedure - #4379

Open
fhan688 wants to merge 1 commit into
apache:mainfrom
fhan688:add-Flink-sys.describe_buckets-procedure
Open

fhan688 wants to merge 1 commit into
apache:mainfrom
fhan688:add-Flink-sys.describe_buckets-procedure

Conversation

@fhan688

@fhan688 fhan688 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #3436

PR #4029 introduced the Admin API and RPC support for retrieving bucket metadata. This pull request exposes that functionality to Flink SQL users through the sys.describe_buckets procedure.

The procedure supports describing all buckets of a table and filtering partitioned tables with a complete or partial partition spec.

Brief change log

  • Add and register the sys.describe_buckets Flink system procedure.
  • Provide two procedure overloads:
    • CALL sys.describe_buckets('database.table')
    • CALL sys.describe_buckets('database.table', 'key=value[/key=value...]')
  • Return the complete bucket metadata:
    • table path and table ID
    • partition ID and partition name
    • bucket ID
    • leader ID and leader epoch
    • bucket epoch
    • replicas and ISR
  • Preserve the existing metadata semantics:
    • partition fields are NULL for non-partitioned tables
    • leader fields are NULL when no leader is available
    • a missing bucket epoch is returned as NULL, while -1 represents missing leader/ISR state
  • Validate table paths and partition specs, including malformed input and duplicate partition keys.
  • Add unit and integration coverage for non-partitioned tables, partitioned tables, partial and complete partition specs, nullable fields, invalid arguments, and propagated Admin API errors.
  • Document the procedure syntax, parameters, result columns, permissions, and usage examples.

Tests

  • DescribeBucketsProcedureTest: 5 tests passed.
  • Flink118ProcedureITCase#testDescribeBuckets passed.
  • Flink22ProcedureITCase#testDescribeBuckets passed.
  • Spotless passed.
  • Checkstyle passed.
  • RAT passed.
  • git diff --check passed.

API and Format

This pull request adds a new Flink SQL system procedure and its result schema.

It does not change the Java Admin API, RPC protocol, server implementation, or storage format. The procedure delegates to the Admin#describeBuckets APIs introduced by PR #4029.

Documentation

Added documentation for sys.describe_buckets to website/docs/engine-flink/procedures.md, including syntax, parameters, output columns, metadata semantics, permission requirements, and examples.

@fhan688 fhan688 closed this Sep 17, 2026
@fhan688 fhan688 reopened this Sep 17, 2026
@fhan688

fhan688 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Hi @swuferhong , this PR is the Flink procedure follow-up to #4029. It exposes the describe-buckets API through sys.describe_buckets and completes #3436. All CI checks are passing now.

Could you please take a look when you have time? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Admin API and sys procedure to describe bucket metadata

1 participant