Skip to content

Align FIPS test plan, requirements, and e2e FIPS coverage, fixed failing operator tests#2011

Open
Elmo33 wants to merge 10 commits into
Altinity:0.27.2from
Elmo33:0.27.2
Open

Align FIPS test plan, requirements, and e2e FIPS coverage, fixed failing operator tests#2011
Elmo33 wants to merge 10 commits into
Altinity:0.27.2from
Elmo33:0.27.2

Conversation

@Elmo33

@Elmo33 Elmo33 commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

This PR aligns the FIPS 140-3 test plan, generated requirements, and e2e operator test coverage.

Main changes:

  • Add the FIPS software test plan under tests/requirements/fips_test_plan.md.
  • Restructure and consolidate FIPS requirements in tests/requirements/fips.md.
  • Regenerate/update tests/requirements/fips.py from the updated requirement definitions.
  • Update FIPS e2e scenarios in test_operator.py to reference the corrected/consolidated requirements.
  • Refine FIPS helper steps in steps_fips.py to better match what is actually testable.
  • Update ACVP-related requirements/tests for the metrics-exporter wrapper behavior and expected algorithm coverage.

Details

This PR makes the three layers consistent:

  1. Test plan

    • Adds the FIPS test plan as an explicit tracked artifact.
    • Documents the intended FIPS coverage and known scope boundaries.
  2. Requirements

    • Consolidates duplicate or overly-specific requirements into broader testable requirements.
    • Clarifies FIPS enforcement behavior such as TLS verification coercion, TLS 1.3 minVersion coercion, IPC secure mode, image policy handling, and operator-managed TLS client scope.
    • Adds/updates connection requirements for operator/exporter communication with Kubernetes API, ClickHouse, and Keeper-related paths.
  3. Tests

    • Updates scenario requirement links to match the revised requirement structure.
    • Improves TLS checks around real endpoints:
      • Kubernetes API :443
      • ClickHouse HTTPS :8443
    • Verifies TLS 1.3 negotiation with approved AES-GCM cipher suites where applicable.
    • Avoids pretending the operator performs runtime TLS client sessions to Keeper when it normally does not.
    • Keeps checks focused on observable behavior instead of synthetic assumptions that do not represent the real operator data path.
  4. FIX

  • fixed test_010035_2 manifest and assertion
  • fail proofed test_010023 by adding wait for pod deployment

Important items to consider before making a Pull Request

Please check items PR complies to:

  • All commits in the PR are squashed. More info
  • The PR is made into dedicated next-release branch, not into master branch1. More info
  • The PR is signed. More info

--

1 If you feel your PR does not affect any Go-code or any testable functionality (for example, PR contains docs only or supplementary materials), PR can be made into master branch, but it has to be confirmed by project's maintainer.

sunsingerus added a commit that referenced this pull request Jun 23, 2026
Resolved all conflicts in favor of our local 0.27.2 work:
- test-058-secret.yaml: kept our regenerated cert (valid to 2126, wildcard SAN)
  over the PR's 2029 cert.
- test_020017 / test_010063 (-client. resolver assertion): kept ours intact.
- steps_fips.py: took the PR's coherent rewrite (our 9182-plaintext intent is
  preserved/improved there); the -X ours auto-graft was discarded.
- Brought in the PR's new content: fips_test_plan.md, consolidated fips.md/fips.py
  (34 reqs, zero dangling refs), new FIPS manifests, util._apply_operator_godebug.

Known test bugs from the PR fixed in a follow-up commit (test_010035_2 false-green,
test_010035_3 livenessProbe breakage).
sunsingerus added a commit that referenced this pull request Jun 23, 2026
PR #2011 added a livenessProbe to test-035-2-sustained-not-ready.yaml and
reworked test_010035_2 to assert kubelet restartCount — a false green: the
operator's sustained-NotReady pod recreation (PR #1998) was asserted nowhere,
and the livenessProbe restored /tmp/ready (breaking test_010035_3's stays-NotReady
premise).

- Restore the manifest to readinessProbe-only (pod stays NotReady so the operator,
  not the kubelet, is what acts).
- test_010035_2: assert the operator recreates the pod (UID change) within the
  sustained-NotReady window, per the actual recovery behavior.
- test_010035_3 (opt-out) now holds: pod stays NotReady, UID unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant