Rewrite/status management - #335
Open
Tanker2020 wants to merge 22 commits into
Open
Conversation
- Node: name, optional NodeFunc, directed edges to upstream deps - Edge: carries optional EdgeFunc to gate dependent start - ResourceNode: embeds Node, adds Manifest/VerifyFunc/DeployMethod - Graph: root-anchored container; AddNode, AddDependency, Topology - Cycle detection on every AddChild call (DFS reachability) - Topology() returns DFS post-order (dependency-first deploy order) Files: rewrite/dag/node.go, rewrite/dag/graph.go Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- CompletionState: Verified/Unverified/Failed/Unstarted node buckets
- DeployCompleted(), VerifyCompleted(), AnyFailed() predicates
- HaltError{Fatal bool}: returned by NodeFunc to signal runner halt
Fatal=true → node lands in Failed, downstreams become Unstarted
Fatal=false → node lands in Unverified (deployed, not yet ready)
Files: rewrite/dag/completion_state.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Python used ThreadPoolExecutor + time.sleep(0.05) busy-poll loop. Go port uses goroutines + buffered results channel; scheduler blocks on select — zero CPU busy-polling. - NewRunner(graph, opts...) with functional options - WithConcurrency(0): serial topology walk, no goroutines (dry-run/test) - WithConcurrency(n): semaphore-capped parallel execution - WithVerifyUpstream(bool): gate dependent start on EdgeFunc result - context.Context cancellation: drains in-flight, marks rest Unstarted - Independent graph branches continue executing after sibling failure (matches Python oper8 intended behaviour; Python had a bug where the serial loop broke early on fatalErr) - stateMap protected by sync.Mutex; scheduler is single writer Files: rewrite/dag/runner.go Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
23 test cases covering:
Graph/Node: empty graph, duplicate node, empty name, cycle detection,
self-loop, topology order, String()
Runner serial: all succeed, empty graph, fatal halt (independent branch
still runs), unverified halt, disabled node, execution order
Runner concurrent: all succeed, fatal halt, independent nodes verified
parallel via start-time spread, race detector stress test
(20 nodes, atomic counter), context cancellation
EdgeFunc: blocks dependent when returns false, allows when true
CompletionState: all predicate combinations
ResourceNode: construction and field access
Concurrency test uses start-time recording rather than wall-clock
total elapsed — CI-safe on slow runners.
Files: rewrite/dag/runner_test.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- Matrix: Go 1.22 and 1.23 - go test -race -count=1 -timeout=60s ./dag/... - go build ./... and go vet ./dag/... - golangci-lint on dag/ package - Triggered on push to rewrite/DAG_Runner and PRs targeting main - working-directory: rewrite (module root) Files: ./.github/workflows/pr1-dag-runner.yml Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Defines the core abstraction all cluster interactions go through. Python (bool, bool) return tuples → Go (changed bool, err error): - success bool dropped; errors are returned as error values - callers use idiomatic `if err != nil` instead of checking two booleans watch_objects Python generator → Go channel: - Watch() returns <-chan WatchEvent; caller ranges over it - Cancelled via context.Context; channel is closed on cancel New types vs Python: - ListOptions struct (replaces positional label_selector/field_selector args) - EventType string constants (ADDED/MODIFIED/DELETED) - WatchEvent struct with Timestamp Files: rewrite/deploymanager/deploymanager.go Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Ports deploy_manager/owner_references.py. - OwnerRef(ownerCR) builds a single ownerReference map entry - ApplyOwnerRef(owner, child) stamps the reference onto child.metadata - No-op when owner == child (same UID) - No-op for cross-namespace references (K8s does not support them) - Idempotent: will not add duplicate entries - blockOwnerDeletion: true; controller field intentionally omitted (matches Python behaviour and StackOverflow rationale in source) Files: rewrite/deploymanager/ownerref.go Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Ports deploy_manager/dry_run_deploy_manager.py. Primary use: unit-testing controllers without a live cluster. Key differences from Python: - Python used nested defaultdict; Go uses typed clusterStore (map[ns][kind][apiVersion][name] → object) - Python RLock on class level; Go sync.RWMutex per instance - Python watch callbacks were registered functions; Go uses buffered channels — consumers range over the channel, cancel via context - Watch channel is closed when ctx is cancelled (no explicit Unregister) - deepCopy via JSON marshal/unmarshal (simple, correct for map[string]any) - matchSelector implements = == != existence operators (sufficient for dry-run tests; full set-based selector is future work) Extra test helpers (not in Python): - GetStored(ns, kind, av, name) — direct store access for assertions - ObjectCount() — total objects in store Files: rewrite/deploymanager/dryrun.go Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
19 test cases covering:
Deploy: create, idempotent re-deploy, field update, owner ref stamping
Get: not found returns nil, found returns deep copy (mutation check)
Delete: existing object, non-existent no-op
List: all objects, label selector filtering
SetStatus: sets status, returns changed=true; error on missing object
Watch: receives ADDED on deploy, DELETED on delete, channel closes
on context cancel (race-detector safe)
OwnerRef: stamps reference, idempotent, cross-namespace skipped
Files: rewrite/deploymanager/dryrun_test.go
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
- Matrix: Go 1.22 and 1.23 - go test -race -count=1 -timeout=60s ./deploymanager/... - go build ./... and go vet ./deploymanager/... - golangci-lint on deploymanager/ package - Triggered on push to rewrite/Deploy_Manager and PRs targeting main Files: .github/workflows/pr2-deploy-manager.yml Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…licationStatus Ports oper8 Python status.py to Go. Reason types: ReadyReason, UpdatingReason, ServiceStatus string constants. MakeApplicationStatus(Options) builds a complete status map: - Ready + Updating conditions from reason/message pairs - External conditions preserved alongside oper8-managed ones - ComponentStatus block from dag.CompletionState (sorted node names) - versions.reconciled / versions.available.versions (IBM CloudPak paths) - <kind>Status field (e.g. customerStatus) when Kind is set UpdateApplicationStatus merges new Options onto existing status, carrying forward current reasons and external conditions when not overridden. GetCondition, GetVersion, StatusChanged helper functions included. Python translation notes: - deepdiff library dropped; StatusChanged uses recursive JSON comparison after stripping lastTransactionTime keys — zero external dependencies - **kwargs replaced by Options struct (compile-time field checking) - aconfig nested_set/nested_get replaced by nestedSet/nestedGet dot-path helpers Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
MakeApplicationStatus: Ready/Updating condition status values for all reason combinations, empty options, external conditions, version fields, componentStatus deployed/verified counts and dependencyGraph, IBM CloudPak <kind>Status (Completed/Failed/InProgress/custom preserved) UpdateApplicationStatus: preserves existing reasons when not overridden, overrides when provided, preserves external conditions and top-level fields StatusChanged: same content + different timestamps not changed, different reason changed, nil inputs, added field GetCondition, GetVersion: found/missing cases UpdatingReason active/inactive matrix (all 6 reasons) ComponentStatus node names sorted alphabetically Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Matrix Go 1.22 and 1.23, race detector, golangci-lint v1.64.8 Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…nentStatus Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
…load config Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Signed-off-by: Nishanth Kolakalapudi <Nishanth.Kol@ibm.com>
Tanker2020
force-pushed
the
rewrite/Status_Management
branch
from
August 11, 2026 08:13
70576d0 to
7efadb8
Compare
Tanker2020
marked this pull request as ready for review
August 13, 2026 16:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the Python
oper8status management layer to Go. Provides helpers for building and updating thestatusblock of oper8-managed Kubernetes custom resources, including theReady/Updatingcondition pair, IBM CloudPak service-status fields, component-status reporting, and a timestamp-aware diff to avoid spurious status patches.Files changed:
rewrite/status/status.go— all public APIrewrite/status/status_test.go— 28 table-driven tests.github/workflows/pr3-status.yml— CIDepends on: PR-1 (
dag/package —CompletionStateandNodeare used incomponentStatusgeneration). Stacked on PR-2 branch but does not importdeploymanager/.What was ported
status.py—make_application_status()status.MakeApplicationStatus(opts Options)status.py—update_application_status()status.UpdateApplicationStatus(current, opts Options)status.py—status_changed()status.StatusChanged(current, proposed)status.py—get_condition()status.GetCondition(condType, status)status.py—get_version()status.GetVersion(status)ReadyReason,UpdatingReason,ServiceStatusenumsstringconstants (same values)Design decisions
**kwargs→OptionsstructPython's
make_application_statusaccepted 12 keyword arguments. Go uses a singleOptionsstruct. Zero values are treated as "not provided" — callers set only the fields they need. This makes call sites self-documenting and prevents argument-order mistakes.deepdiff→ JSON round-trip with timestamp strippingPython's
status_changed()used thedeepdifflibrary, excluding timestamp keys viaexclude_obj_callback. Go replaces this with a recursivestripTimestamps()helper that deep-copies the map while droppinglastTransactionTimekeys, then compares viajson.Marshal. No external dependency.TimestampKey = "lastTransactionTime"(notlastTransitionTime)The Python source uses the non-standard key
lastTransactionTime(oper8-specific, not the standard KuberneteslastTransitionTime). The Go port preserves this exactly to maintain wire compatibility with existing CR status fields.config.operator_versionnot injected inUpdateApplicationStatusPython's
update_application_statusautomatically stampedconfig.operator_versioninto every status update. Theconfigpackage is not yet ported (future PR-5).UpdateApplicationStatusacceptsopts.OperatorVersionexplicitly — callers that need it can pass it; nothing is silently injected.update_resource_statusnot portedPython's
update_resource_statuscombinedget_object_current_state+update_application_status+set_statusinto one helper. This convenience function belongs in the reconcile layer (PR-5) which has access to a liveDeployManager. It is not included here to keep this package free ofdeploymanagerdependency.Bug fix:
verifiedComponentskey was missingThe initial port omitted the
verifiedComponentslist fromcomponentStatus— the Python_make_component_statefunction emits it, and downstream consumers (dashboards, tooling) read it. Fixed and covered by a regression test.Test coverage (28 tests)
MakeApplicationStatusUpdateApplicationStatusStatusChangedGetConditionGetVersionUpdatingReasonactive/inactive matrixcomponentStatussortingverifiedComponentsregressionAll tests run with
-race.