Skip to content

fix(ctrace): accept default TB captures and report skipped payload - #2624

Merged
thorstendb-ARM merged 10 commits into
mainfrom
investigate/ctrace-tb-rejection
Sep 22, 2026
Merged

thorstendb-ARM merged 10 commits into
mainfrom
investigate/ctrace-tb-rejection

Conversation

@thorstendb-ARM

Copy link
Copy Markdown
Contributor

Fixes

  • TB/TB_* captures rejected when the optional trace-format declaration is missing or null.
  • Premature failure on initial formatter payload without an assigned source ID.
  • Missing visibility of skipped payload and a summary that described diagnostic records as decoded events.

Changes

  • Resolve the input format before route normalization: TB/TB_* defaults to formatted CoreSight, SWO to
    unformatted ITM. Explicit declarations retain precedence; input selection remains unambiguous.
  • Tolerate initial unassigned formatter payload and account for payload skipped due to missing, NULL,
    reserved, or unconfigured source IDs, plus initial routed bytes before real hardware synchronization.
  • Report skipped bytes consistently as CLI Info and timestamp-free CSV info rows, independently of
    --type and --stream filters. These annotations do not create synthetic routes or CTF events.
  • Keep missing synchronization on an observed configured route as an end-of-input Error with non-zero exit,
    retaining completed diagnostics and healthy-route outputs. Genuine framing failures still abort outputs.
  • Use packet-neutral skip wording and a summary of processed input bytes and trace/diagnostic records,
    rather than claiming that every record represents a decoded payload event.
  • Extend regression coverage and update README, architecture MD/SVG, constraints, and output documentation.

Validation

  • Native macOS arm64 Release build passed.

  • All seven ctrace CTest groups passed: 377 unit tests passed with five platform-specific skips,
    37 integration tests passed, plus fixture integrity and executable/manifest smoke checks.

  • Regression coverage includes channel defaults and overrides, incomplete frames, exact byte accounting,
    chunk boundaries, late/missing synchronization, independent routes, filtering, output failures,
    legacy raw SWO golden outputs, and the reconstructed multi-source TB fixture.

  • Three additional real TB captures were checked locally with --csv:

    Raw size Result
    64 bytes No hardware SYNC: four skip Infos and one Error; exit 1.
    4,160 bytes Hardware SYNC only at the end: four skip Infos; no payload events; exit 0.
    4,096 bytes Hardware SYNC only at the end: three skip Infos; no payload events; exit 0.

    The latter two captures have no routed payload after their first SYNC; they are not successful PC-sample decodes.

  • Changed Markdown files, SVG XML syntax, and git diff --check passed.

Scope and limitations

  • No raw bytes are rewritten, no synthetic synchronization is injected, and no heuristic recovery CLI option is added.
  • Counts describe deformatted payload bytes, excluding formatter-control bytes; this is not a general accounting
    mechanism for every possible decoder-recovery loss.
  • Exactly one input remains required; formatted input still uses complete 16-byte memory-aligned frames.
  • trace-format remains a provisional ctrace-private field, not a newly standardized producer requirement.
  • No additional protocol decoders, dependency updates, or repository-global workflow changes.
  • Cross-platform CI and coverage checks will run on this PR; local validation above is macOS arm64.

Checklist

  • 🤖 This change is covered by unit and integration tests.
  • 🤹 Manual testing has been performed with real TB captures.
  • 🛡️ Security impacts considered: bounded accounting, unchanged input, and no invented synchronization.
  • 📖 Documentation updates are complete.
  • 🧠 Third-party dependencies and TPIP reviewed; no changes required.

Resolve absent trace-format from the selected input before route normalization: formatted for TB channels and unformatted for SWO, while retaining explicit overrides.

Allow initial unassigned formatter payload, account for skipped source and initial unsynchronized bytes, and retain timestamp-free CSV info rows independently of event filters. Keep missing synchronization as a separate decoding error and use packet-neutral skip wording.

Add regression coverage and update architecture, constraints, and output documentation. Validated with the Release build, all seven ctrace CTest groups, and the Jens TB capture.
Report processed input bytes and trace/diagnostic records instead of claiming that every record is a decoded event. Clarify that counters are collected before output filtering.

Extend integration assertions for info-only and error-only input and preserve the counters for captured SWO traces. Validated with the Release build, all seven ctrace CTest groups, and three real TB captures.
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Test Results

    6 files      6 suites   6s ⏱️
  454 tests   454 ✅  0 💤 0 ❌
1 362 runs  1 348 ✅ 14 💤 0 ❌

Results for commit 68004cd.

♻️ This comment has been updated with latest results.

Comment thread tools/ctrace/src/decode/OpenCsdFormattedItmSession.cpp Fixed
Use size_t for skipped-source array traversal and narrow only at the
source-ID interface.

Remove source-ID-loss and replay-accounting paths that are unreachable
with the configured OpenCSD frontend, and document the invariant.
Cover normalized setup warnings, optional skip callbacks, invalid route
configuration, and source-ID retention across frame and route resets.

Validation: 100% source-line coverage (6508/6508) with the unchanged
Linux CI gate; 8/8 Linux and 7/7 macOS ctrace test groups pass.
Preserve native OpenCSD packet context and report explicit recovery outcomes. Keep committed CSV rows with an unfiltered abort marker after fatal decode failures while removing incomplete CTF output.

Extend regression coverage, reconcile documentation and the architecture diagram, and publish the concise single-source to multi-ID design record.
Remove unused DecodeConsumers::abortOutputs and the now-unneeded TraceOutputLifecycle::abort forwarding method after fatal decoding moved to finishOutputs with TraceDecodeAbort.

Exercise fatal-output finalization and destructor cleanup in the corresponding tests. This removes the cppcheck unusedFunction finding without suppressions or workflow changes.

@jreineckearm jreineckearm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two comments from code review. Rest of the code looks good.
Taking for a manual test now.

Comment thread tools/ctrace/docs/multi-source-design.md Outdated
Comment thread tools/ctrace/docs/THIRD_PARTY_NOTICES.md Outdated
jreineckearm
jreineckearm previously approved these changes Sep 22, 2026

@jreineckearm jreineckearm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing looks good, I'd be fine with keeping the trace-formatter provision in for the time being. But as mentioned in my comment I don't see it becoming part of the spec.

Document trace-format as a private temporary override without assuming standardization, and align the input-contract follow-up with cmsis-toolbox #699. Move packaging and SHA256SUMS details from third-party notices to the architecture documentation.
Preserve TB byte-skip diagnostics while incorporating PC-sampling markers from #2629. Resolve fixture inventory, route-selection test and documentation conflicts. Verify the marker-only CSV output including NULL-source padding annotations and remove obsolete marker limitations.
@codecov

codecov Bot commented Sep 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.67%. Comparing base (071e08f) to head (68004cd).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2624      +/-   ##
==========================================
+ Coverage   74.51%   74.67%   +0.16%     
==========================================
  Files         197      197              
  Lines       32562    32778     +216     
  Branches    19036    19152     +116     
==========================================
+ Hits        24262    24478     +216     
+ Misses       6000     5997       -3     
- Partials     2300     2303       +3     
Flag Coverage Δ
ctrace-cov 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
tools/ctrace/src/control/DecodeConsumers.cpp 100.00% <100.00%> (ø)
tools/ctrace/src/control/FileDecodeJob.cpp 100.00% <100.00%> (ø)
tools/ctrace/src/control/TraceDirectoryJob.cpp 100.00% <100.00%> (ø)
tools/ctrace/src/decode/DecodePipeline.cpp 100.00% <100.00%> (ø)
tools/ctrace/src/decode/DecodePipeline.h 100.00% <ø> (ø)
tools/ctrace/src/decode/OpenCsdErrorController.cpp 100.00% <100.00%> (ø)
...s/ctrace/src/decode/OpenCsdFormattedItmSession.cpp 100.00% <100.00%> (ø)
...ols/ctrace/src/decode/OpenCsdFormattedItmSession.h 100.00% <100.00%> (ø)
tools/ctrace/src/decode/OpenCsdItmDecoder.cpp 100.00% <100.00%> (ø)
tools/ctrace/src/decode/OpenCsdItmDecoder.h 100.00% <ø> (ø)
... and 12 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jreineckearm jreineckearm left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates, reviewed them. No need to retest as pretty much all was in documentation files. Good to merge.

@thorstendb-ARM
thorstendb-ARM merged commit f46d68b into main Sep 22, 2026
28 checks passed
@thorstendb-ARM
thorstendb-ARM deleted the investigate/ctrace-tb-rejection branch September 22, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants