chore: Consolidate error enums and use expect where possible

[adwk67]

Description

Part of stackabletech/issues#883

One smoke test is failing.

Re-run:
With deactivated chaos-monkey step in one tests:

--- PASS: kuttl (783.85s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_hadoop-3.5.0_zookeeper-3.9.4_zookeeper-latest-3.9.5_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal_openshift-false (320.93s)
        --- PASS: kuttl/harness/smoke_hadoop-3.5.0_zookeeper-3.9.5_zookeeper-latest-3.9.5_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal_openshift-false (187.43s)
        --- PASS: kuttl/harness/smoke_hadoop-3.5.0_zookeeper-3.9.5_zookeeper-latest-3.9.5_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable_openshift-false (141.62s)
        --- PASS: kuttl/harness/smoke_hadoop-3.5.0_zookeeper-3.9.4_zookeeper-latest-3.9.5_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable_openshift-false (133.87s)
PASS

With all steps for the affected test:

--- PASS: kuttl (676.54s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_hadoop-3.5.0_zookeeper-3.9.5_zookeeper-latest-3.9.5_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-cluster-internal_openshift-false (676.53s)
PASS

N.B. the test was fixed (flakiness due to a race condition) in this PR although the cause predates it: we may want to fix this in a separate PR or fix it at the haadmin probe level in the code instead.

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• CRD changes approved
• CRD documentation for all fields, following the style guide.
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)
• Changes need to be "offline" compatible
• Links to generated (nightly) docs added
• Release note snippet added

Reviewer

• Code contains useful comments
• (Integration-)Test cases added
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Links to generated (nightly) docs added
• Release note snippet added
• Add type/deprecation label & add to the deprecation schedule
• Add type/experimental label & add to the experimental features tracker

[siegfriedweber]

Verifying this expectation took me a while. It is not obvious, and that alone is an argument for keeping the error handling here: if it takes this much digging to confirm, we may well have overlooked something.

As it turns out, the PVC names are in fact not unique:

dataNodes:
  roleGroups:
    default:
      config:
        resources:
          storage:
            data:
              count: 3
              capacity: 1Gi
              hdfsStorageType: Disk
            data-2:
              count: 3
              capacity: 1Gi
              hdfsStorageType: Disk

crd::storage::pvc_names turns the user-provided (!) storage name data into the PVC names data, data-1 and data-2, and the equally user-provided name data-2 into data-2, data-2-1 and data-2-2. Two PersistentVolumeClaims named data-2 therefore end up in the StatefulSet.

add_volume_mounts still does not return an error, because both VolumeMounts have the same content, and in that case the duplicate is silently dropped:

- mountPath: /stackable/data/data-2
  name: data-2