glusterfs: call gluster --xml directly, drop bundled gstatus dependency by thieman · Pull Request #25211 · DataDog/integrations-core

thieman · 2026-09-14T18:15:30Z

What does this PR do?

Rewrites the glusterfs integration to call the gluster CLI's native --xml output directly, removing the dependency on the bundled gstatus binary (and the vendored glustercli/glusterlib GPL sources) that the Datadog Agent currently ships in deps/gstatus/.

A new stdlib-only module datadog_checks/glusterfs/gluster_xml.py parses gluster --xml volume info, gluster --xml volume status all detail, gluster --xml pool list, gluster --xml volume heal <vol> info, and gluster --version into the same intermediate dict shape metrics.py already consumes. The emitted metric names, tags, and service checks are unchanged.

This is the integrations-core half of removing gstatus entirely. A follow-up PR in datadog-agent will delete deps/gstatus/ and its Bazel/renovate references once this version of the integration is pinned.

Motivation

Eliminate the vendored GPL code and the licensing complication it creates (glusterlib is GPLv3, not the GPLv2 the deps/gstatus LICENSE claims).
Drop the bundled gstatus binary shipped with the Agent (previously an 8 MiB frozen PyInstaller bundle; more recently a vendored Python wrapper script). Either way it is dead weight the Agent no longer needs to ship.
The gluster CLI is already a prerequisite (glusterfs-server) on any node running this check, and the vendored glustercli already assumed gluster was on PATH.

Behavior changes

New gluster_command instance option (list, default ["gluster"]) replaces the gstatus_path init_config option. gstatus_path is now deprecated and ignored (a warning is logged if it is still set). gluster_command supports custom paths and wrappers such as ["docker", "exec", "<container>", "gluster"].
glusterfs.volume.used.percent is now a numeric gauge computed by the parser, replacing the human-readable v_size/v_size_used strings and the PARSE_METRICS re-parse hack in metrics.py. The metric name is unchanged.
Node count and active nodes now come from gluster pool list (which includes the local node) rather than gstatus's peer.pool() indirection.
Quota and snapshot sub-commands are no longer collected; no metrics depended on them (snapshot_count comes from volume info).
Each gluster call has a 30s timeout so a hung command cannot block the agent.
Self-heal info is collected best-effort: if volume heal <vol> info fails or times out for a volume, the check logs a warning and yields an empty heal list for that volume instead of aborting. This mirrors how the bundled gstatus treated heal ("Note: Unable to get self-heal status for one or more volumes"). Cluster/volume/brick metrics still emit.

Validation

Validated against a live 2-node GlusterFS 7.1 cluster (the same docker-compose the E2E tests use) by running the actual GlusterfsCheck against it via docker exec gluster-node-1 gluster:

volume info, volume status all detail, pool list, and gluster --version parsing all verified against real XML output.
All cluster/volume/brick/subvol metrics and the three service checks emit correctly against the live cluster.
Graceful heal degradation verified against the live cluster: volume heal info hangs under local arm64 emulation (a known gluster-on-QEMU issue; works on native amd64 in CI), and the check still emits every metric except glusterfs.heal_info.entries.count, exactly as designed. A unit test (test_check_emits_metrics_when_heal_fails) covers this path.

Unit tests (3) and lint pass locally. Integration and E2E tests run against the same docker-compose cluster in CI; the E2E readiness check now also waits for self-heal info to respond before running.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

… binary Replace the vendored `gstatus` wrapper with direct `gluster --xml` calls and a new stdlib-only XML parser (`gluster_xml.py`). The check now runs `gluster volume info`, `gluster volume status all detail`, `gluster pool list`, `gluster volume heal <vol> info`, and `gluster --version` itself, removing the dependency on the GPL-licensed gstatus/glustercli/glusterlib stack that is shipped with the Agent. This is a prerequisite for dropping `deps/gstatus/` from the datadog-agent repo (the bundled binary and its vendored GPL sources) in a follow-up change. The emitted metric names, tags, and service checks are unchanged. Behavior changes: - New `gluster_command` instance option (list, default `["gluster"]`) replaces the `gstatus_path` init_config option, which is now deprecated and ignored (a warning is logged if it is still set). `gluster_command` supports custom paths and wrappers such as `["docker", "exec", "<container>", "gluster"]`. - `glusterfs.volume.used.percent` is now a numeric gauge computed by the parser, replacing the human-readable "v_size"/"v_size_used" strings and the `PARSE_METRICS` re-parse hack in `metrics.py`. The metric name is unchanged. - Node count and active nodes now come from `gluster pool list` (which includes the local node) instead of gstatus's `peer.pool()` indirection. - Quota and snapshot sub-commands are no longer collected; no metrics depended on them (`snapshot_count` comes from `volume info`). Tests now feed XML fixtures (`tests/fixtures/gluster/`) through a mock of `_run_gluster` instead of the gstatus JSON fixture. The E2E docker-compose setup drops the gstatus binary download from its Dockerfile and polls `gluster --xml volume status all detail` for readiness.

github-actions · 2026-09-14T18:16:05Z

⚠️ Major version bump
The changelog type changed or removed was used in this Pull Request, so the next release will bump major version. Please make sure this is a breaking change, or use the fixed or added type instead.

cit-pr-commenter-54b7da · 2026-09-14T18:18:41Z

evalya-impact-summary

evalya impact analysis

Impact analysis: 0 selected, 0 skipped (of 0 test tasks)
Publish tasks:   2 (always emitted)
Diff (22 files):
  glusterfs/README.md
  glusterfs/assets/configuration/spec.yaml
  glusterfs/changelog.d/25211.changed
  glusterfs/datadog_checks/glusterfs/check.py
  glusterfs/datadog_checks/glusterfs/config_models/__init__.py
  glusterfs/datadog_checks/glusterfs/config_models/defaults.py
  glusterfs/datadog_checks/glusterfs/config_models/instance.py
  glusterfs/datadog_checks/glusterfs/config_models/shared.py
  glusterfs/datadog_checks/glusterfs/data/conf.yaml.example
  glusterfs/datadog_checks/glusterfs/gluster_xml.py
  glusterfs/datadog_checks/glusterfs/metrics.py
  glusterfs/tests/common.py
  glusterfs/tests/conftest.py
  glusterfs/tests/docker/Dockerfile
  glusterfs/tests/fixtures/gluster/heal_info.xml
  glusterfs/tests/fixtures/gluster/pool_list.xml
  glusterfs/tests/fixtures/gluster/version.txt
  glusterfs/tests/fixtures/gluster/volume_info.xml
  glusterfs/tests/fixtures/gluster/volume_status.xml
  glusterfs/tests/fixtures/gstatus.txt
  ... and 2 more

Debug a specific task: evalya plan impact --path <path> --task <task>

Learn more about CI impact filtering

datadog-datadog-prod-us1-2 · 2026-09-14T18:18:41Z

✨ Unblock PR with BitsAI

⚠️ Warnings

❌ Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 1 Pipeline job failed

Validate repository | Run Validations / Validate — 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 70.73%
• Overall Coverage: 74.09%

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: fd55545 | Docs | View more details | Give us feedback!}

dd-octo-sts · 2026-09-14T18:21:16Z

Disk usage change

Commit fd55545 compared against f0a455e.

Uncompressed

linux-aarch64	linux-x86_64	macos-aarch64	macos-x86_64	windows-x86_64
🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB

Details

Name	Version	Type	linux-aarch64	linux-x86_64	macos-aarch64	macos-x86_64	windows-x86_64
glusterfs	3.5.0	Integration	🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB	🔺 +17.09 KiB

Compressed

linux-aarch64	linux-x86_64	macos-aarch64	macos-x86_64	windows-x86_64
🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB

Details

Name	Version	Type	linux-aarch64	linux-x86_64	macos-aarch64	macos-x86_64	windows-x86_64
glusterfs	3.5.0	Integration	🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB	🔺 +4.7 KiB

thieman · 2026-09-14T18:27:15Z

  - template: instances
    options:
-    - name: use_sudo
+    - name: gluster_command


How did gstatus originally determine where to call gluster, wasn't it shelling out to it? Ideally we would not add this configuration as it implies that some currently-working customer installs would break without setting this. Is it possible to determine the gluster location the same way gstatus is? Or maybe it just relies on it being in the PATH?

thieman · 2026-09-14T18:33:17Z

+
+The XML schema parsed here is Gluster's own documented ``--xml`` output format;
+the parsing implementation is original and does not derive from the upstream
+GPL-licensed ``gstatus`` sources.


Remove this last paragraph in the comment, unnecessary

…meout A hung or unresponsive self-heal daemon (gluster volume heal <vol> info) must not suppress the cluster/volume/brick metrics or hang the agent. Each gluster call now has a 30s timeout, and heal collection is wrapped so a per-volume heal failure logs a warning and yields an empty heal list instead of aborting the whole check. This mirrors how the upstream gstatus treated heal as best-effort ("Note: Unable to get self-heal status for one or more volumes"). Validated against a live 2-node GlusterFS 7.1 cluster: the check emits all cluster/volume/brick/subvol metrics and service checks even when volume heal info hangs. Added a unit test covering the heal-failure graceful-degradation path. The E2E readiness check now also waits for self-heal info to respond before running.

dd-octo-sts · 2026-09-15T13:54:18Z

✅ Dispatcher tests · passed

Dispatcher beta: informational only

Dispatcher is running alongside existing CI while we validate it. You can ignore this report and its statuses. Existing CI remains the merge signal.

2/2 jobs
✅ 2 passed · ❌ 0 failed

Batches

Batch	State	Jobs	Workflow
`batch-01`	✅ passed	2/2	run 34978048792

_{Dispatcher finished on b4020fc — GitHub Run.}

dd-octo-sts · 2026-09-15T13:54:43Z

Validation Report

Validation	Description	Status
`license-headers`	Validate Python files have proper license headers	❌
`models`	Validate configuration data models match spec.yaml	❌

Run ddev validate all changed --fix to attempt to auto-fix supported validations.

Passed validations (19)

Validation	Description	Status
`agent-reqs`	Verify check versions match the Agent requirements file	✅
`ci`	Validate CI configuration and code coverage settings	✅
`codeowners`	Validate every integration has a CODEOWNERS entry	✅
`config`	Validate default configuration files against spec.yaml	✅
`dep`	Verify dependency pins are consistent and Agent-compatible	✅
`http`	Validate integrations use the HTTP wrapper correctly	✅
`imports`	Validate check imports do not use deprecated modules	✅
`integration-style`	Validate check code style conventions	✅
`jmx-metrics`	Validate JMX metrics definition files and config	✅
`labeler`	Validate PR labeler config matches integration directories	✅
`legacy-signature`	Validate no integration uses the legacy Agent check signature	✅
`licenses`	Validate third-party license attribution list	✅
`metadata`	Validate metadata.csv metric definitions	✅
`openmetrics`	Validate OpenMetrics integrations disable the metric limit	✅
`package`	Validate Python package metadata and naming	✅
`qa-label`	Validate the pull request declares whether it needs QA for the next Agent release	✅
`readmes`	Validate README files have required sections	✅
`saved-views`	Validate saved view JSON file structure and fields	✅
`version`	Validate version consistency between package and changelog	✅

View full run

thieman · 2026-09-15T18:10:51Z

Closing this as the alternative implementation in DataDog/datadog-agent#56202 won't cause any customer breakages, assuming we can get it past OSPO review

thieman added the qa/required QA is required for this PR and will generate a QA card label Sep 14, 2026

temporal-github-worker-1 Bot added agent/review-requested docs/review-requested ecosystems/review-requested product/review-requested labels Sep 14, 2026

glusterfs: add changelog entry for #25211

4e59809

dd-octo-sts Bot added documentation integration/glusterfs labels Sep 14, 2026

thieman mentioned this pull request Sep 14, 2026

Replace frozen gstatus binary with from-source script (~8 MiB savings) DataDog/datadog-agent#56202

Open

9 tasks

thieman commented Sep 14, 2026

View reviewed changes

thieman closed this Sep 15, 2026

Navigation Menu

Uh oh!

glusterfs: call gluster --xml directly, drop bundled gstatus dependency - #25211

glusterfs: call gluster --xml directly, drop bundled gstatus dependency#25211
thieman wants to merge 3 commits into
masterfrom
thieman/gluster-direct-xml

thieman commented Sep 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

cit-pr-commenter-54b7da Bot commented Sep 14, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Sep 14, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

Uh oh!

dd-octo-sts Bot commented Sep 14, 2026 •

edited

Loading

Uh oh!

thieman Sep 14, 2026

Uh oh!

thieman Sep 14, 2026

Uh oh!

dd-octo-sts Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

dd-octo-sts Bot commented Sep 15, 2026

Uh oh!

thieman commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

thieman commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Behavior changes

Validation

Review checklist (to be filled by reviewers)

Uh oh!

github-actions Bot commented Sep 14, 2026

Uh oh!

cit-pr-commenter-54b7da Bot commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

evalya-impact-summary

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Sep 14, 2026 • edited by datadog-prod-us1-5 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

dd-octo-sts Bot commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Disk usage change

Uncompressed

Compressed

Uh oh!

thieman Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

thieman Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

dd-octo-sts Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Dispatcher tests · passed

Batches

Uh oh!

dd-octo-sts Bot commented Sep 15, 2026

Validation Report

Uh oh!

thieman commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

thieman commented Sep 14, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Sep 14, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Sep 14, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

dd-octo-sts Bot commented Sep 14, 2026 •

edited

Loading

dd-octo-sts Bot commented Sep 15, 2026 •

edited

Loading