Skip to content

Rewrite/status management - #335

Open
Tanker2020 wants to merge 22 commits into
IBM:mainfrom
Tanker2020:rewrite/Status_Management
Open

Rewrite/status management#335
Tanker2020 wants to merge 22 commits into
IBM:mainfrom
Tanker2020:rewrite/Status_Management

Conversation

@Tanker2020

Copy link
Copy Markdown

Summary

Ports the Python oper8 status management layer to Go. Provides helpers for building and updating the status block of oper8-managed Kubernetes custom resources, including the Ready/Updating condition pair, IBM CloudPak service-status fields, component-status reporting, and a timestamp-aware diff to avoid spurious status patches.

Files changed:

  • rewrite/status/status.go — all public API
  • rewrite/status/status_test.go — 28 table-driven tests
  • .github/workflows/pr3-status.yml — CI

Depends on: PR-1 (dag/ package — CompletionState and Node are used in componentStatus generation). Stacked on PR-2 branch but does not import deploymanager/.


What was ported

Python Go equivalent
status.pymake_application_status() status.MakeApplicationStatus(opts Options)
status.pyupdate_application_status() status.UpdateApplicationStatus(current, opts Options)
status.pystatus_changed() status.StatusChanged(current, proposed)
status.pyget_condition() status.GetCondition(condType, status)
status.pyget_version() status.GetVersion(status)
ReadyReason, UpdatingReason, ServiceStatus enums typed string constants (same values)

Design decisions

**kwargsOptions struct

Python's make_application_status accepted 12 keyword arguments. Go uses a single Options struct. Zero values are treated as "not provided" — callers set only the fields they need. This makes call sites self-documenting and prevents argument-order mistakes.

deepdiff → JSON round-trip with timestamp stripping

Python's status_changed() used the deepdiff library, excluding timestamp keys via exclude_obj_callback. Go replaces this with a recursive stripTimestamps() helper that deep-copies the map while dropping lastTransactionTime keys, then compares via json.Marshal. No external dependency.

TimestampKey = "lastTransactionTime" (not lastTransitionTime)

The Python source uses the non-standard key lastTransactionTime (oper8-specific, not the standard Kubernetes lastTransitionTime). The Go port preserves this exactly to maintain wire compatibility with existing CR status fields.

config.operator_version not injected in UpdateApplicationStatus

Python's update_application_status automatically stamped config.operator_version into every status update. The config package is not yet ported (future PR-5). UpdateApplicationStatus accepts opts.OperatorVersion explicitly — callers that need it can pass it; nothing is silently injected.

update_resource_status not ported

Python's update_resource_status combined get_object_current_state + update_application_status + set_status into one helper. This convenience function belongs in the reconcile layer (PR-5) which has access to a live DeployManager. It is not included here to keep this package free of deploymanager dependency.

Bug fix: verifiedComponents key was missing

The initial port omitted the verifiedComponents list from componentStatus — the Python _make_component_state function emits it, and downstream consumers (dashboards, tooling) read it. Fixed and covered by a regression test.


Test coverage (28 tests)

Area Tests
MakeApplicationStatus ReadyStable, ReadyInProgress, ReadyErrored, no conditions, external conditions, version fields, componentStatus counts, componentStatus dep-graph, kind service status (Completed/Failed/InProgress/custom not overwritten)
UpdateApplicationStatus preserves existing reasons, overrides reason, preserves external conditions, preserves external status fields
StatusChanged same content different timestamp (false), different reason (true), nil inputs, added field
GetCondition found, not found
GetVersion present, missing
UpdatingReason active/inactive matrix all 6 reasons
componentStatus sorting alphabetical node order
verifiedComponents regression key present with correct count

All tests run with -race.

- Node: name, optional NodeFunc, directed edges to upstream deps
- Edge: carries optional EdgeFunc to gate dependent start
- ResourceNode: embeds Node, adds Manifest/VerifyFunc/DeployMethod
- Graph: root-anchored container; AddNode, AddDependency, Topology
- Cycle detection on every AddChild call (DFS reachability)
- Topology() returns DFS post-order (dependency-first deploy order)

Files: rewrite/dag/node.go, rewrite/dag/graph.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- CompletionState: Verified/Unverified/Failed/Unstarted node buckets
- DeployCompleted(), VerifyCompleted(), AnyFailed() predicates
- HaltError{Fatal bool}: returned by NodeFunc to signal runner halt
  Fatal=true  → node lands in Failed, downstreams become Unstarted
  Fatal=false → node lands in Unverified (deployed, not yet ready)

Files: rewrite/dag/completion_state.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Python used ThreadPoolExecutor + time.sleep(0.05) busy-poll loop.
Go port uses goroutines + buffered results channel; scheduler blocks
on select — zero CPU busy-polling.

- NewRunner(graph, opts...) with functional options
- WithConcurrency(0): serial topology walk, no goroutines (dry-run/test)
- WithConcurrency(n): semaphore-capped parallel execution
- WithVerifyUpstream(bool): gate dependent start on EdgeFunc result
- context.Context cancellation: drains in-flight, marks rest Unstarted
- Independent graph branches continue executing after sibling failure
  (matches Python oper8 intended behaviour; Python had a bug where the
  serial loop broke early on fatalErr)
- stateMap protected by sync.Mutex; scheduler is single writer

Files: rewrite/dag/runner.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
23 test cases covering:
  Graph/Node:  empty graph, duplicate node, empty name, cycle detection,
               self-loop, topology order, String()
  Runner serial: all succeed, empty graph, fatal halt (independent branch
               still runs), unverified halt, disabled node, execution order
  Runner concurrent: all succeed, fatal halt, independent nodes verified
               parallel via start-time spread, race detector stress test
               (20 nodes, atomic counter), context cancellation
  EdgeFunc:    blocks dependent when returns false, allows when true
  CompletionState: all predicate combinations
  ResourceNode: construction and field access

Concurrency test uses start-time recording rather than wall-clock
total elapsed — CI-safe on slow runners.

Files: rewrite/dag/runner_test.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- Matrix: Go 1.22 and 1.23
- go test -race -count=1 -timeout=60s ./dag/...
- go build ./... and go vet ./dag/...
- golangci-lint on dag/ package
- Triggered on push to rewrite/DAG_Runner and PRs targeting main
- working-directory: rewrite (module root)

Files: ./.github/workflows/pr1-dag-runner.yml
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Defines the core abstraction all cluster interactions go through.

Python (bool, bool) return tuples → Go (changed bool, err error):
- success bool dropped; errors are returned as error values
- callers use idiomatic `if err != nil` instead of checking two booleans

watch_objects Python generator → Go channel:
- Watch() returns <-chan WatchEvent; caller ranges over it
- Cancelled via context.Context; channel is closed on cancel

New types vs Python:
- ListOptions struct (replaces positional label_selector/field_selector args)
- EventType string constants (ADDED/MODIFIED/DELETED)
- WatchEvent struct with Timestamp

Files: rewrite/deploymanager/deploymanager.go

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Ports deploy_manager/owner_references.py.

- OwnerRef(ownerCR) builds a single ownerReference map entry
- ApplyOwnerRef(owner, child) stamps the reference onto child.metadata
  - No-op when owner == child (same UID)
  - No-op for cross-namespace references (K8s does not support them)
  - Idempotent: will not add duplicate entries
- blockOwnerDeletion: true; controller field intentionally omitted
  (matches Python behaviour and StackOverflow rationale in source)

Files: rewrite/deploymanager/ownerref.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Ports deploy_manager/dry_run_deploy_manager.py.

Primary use: unit-testing controllers without a live cluster.

Key differences from Python:
- Python used nested defaultdict; Go uses typed clusterStore
  (map[ns][kind][apiVersion][name] → object)
- Python RLock on class level; Go sync.RWMutex per instance
- Python watch callbacks were registered functions; Go uses buffered
  channels — consumers range over the channel, cancel via context
- Watch channel is closed when ctx is cancelled (no explicit Unregister)
- deepCopy via JSON marshal/unmarshal (simple, correct for map[string]any)
- matchSelector implements = == != existence operators (sufficient for
  dry-run tests; full set-based selector is future work)

Extra test helpers (not in Python):
- GetStored(ns, kind, av, name) — direct store access for assertions
- ObjectCount() — total objects in store

Files: rewrite/deploymanager/dryrun.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
19 test cases covering:
  Deploy:     create, idempotent re-deploy, field update, owner ref stamping
  Get:        not found returns nil, found returns deep copy (mutation check)
  Delete:     existing object, non-existent no-op
  List:       all objects, label selector filtering
  SetStatus:  sets status, returns changed=true; error on missing object
  Watch:      receives ADDED on deploy, DELETED on delete, channel closes
              on context cancel (race-detector safe)
  OwnerRef:   stamps reference, idempotent, cross-namespace skipped

Files: rewrite/deploymanager/dryrun_test.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- Matrix: Go 1.22 and 1.23
- go test -race -count=1 -timeout=60s ./deploymanager/...
- go build ./... and go vet ./deploymanager/...
- golangci-lint on deploymanager/ package
- Triggered on push to rewrite/Deploy_Manager and PRs targeting main

Files: .github/workflows/pr2-deploy-manager.yml
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…licationStatus

Ports oper8 Python status.py to Go.

Reason types: ReadyReason, UpdatingReason, ServiceStatus string constants.

MakeApplicationStatus(Options) builds a complete status map:
- Ready + Updating conditions from reason/message pairs
- External conditions preserved alongside oper8-managed ones
- ComponentStatus block from dag.CompletionState (sorted node names)
- versions.reconciled / versions.available.versions (IBM CloudPak paths)
- <kind>Status field (e.g. customerStatus) when Kind is set

UpdateApplicationStatus merges new Options onto existing status,
carrying forward current reasons and external conditions when not overridden.

GetCondition, GetVersion, StatusChanged helper functions included.

Python translation notes:
- deepdiff library dropped; StatusChanged uses recursive JSON comparison
  after stripping lastTransactionTime keys — zero external dependencies
- **kwargs replaced by Options struct (compile-time field checking)
- aconfig nested_set/nested_get replaced by nestedSet/nestedGet dot-path helpers

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
MakeApplicationStatus: Ready/Updating condition status values for all
  reason combinations, empty options, external conditions, version fields,
  componentStatus deployed/verified counts and dependencyGraph, IBM CloudPak
  <kind>Status (Completed/Failed/InProgress/custom preserved)

UpdateApplicationStatus: preserves existing reasons when not overridden,
  overrides when provided, preserves external conditions and top-level fields

StatusChanged: same content + different timestamps not changed, different
  reason changed, nil inputs, added field

GetCondition, GetVersion: found/missing cases
UpdatingReason active/inactive matrix (all 6 reasons)
ComponentStatus node names sorted alphabetically

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Matrix Go 1.22 and 1.23, race detector, golangci-lint v1.64.8

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…nentStatus

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…load config

Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
@Tanker2020
Tanker2020 force-pushed the rewrite/Status_Management branch from 70576d0 to 7efadb8 Compare August 11, 2026 08:13
@Tanker2020
Tanker2020 marked this pull request as ready for review August 13, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant