Standardize APIScan registration and SDL break policy in OneBranch pipelines#4462
Standardize APIScan registration and SDL break policy in OneBranch pipelines#4462paulmedynski wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Modernizes the OneBranch official and non-official pipelines’ SDL/APIScan configuration to be more standardized and compatible with newer Windows container images, focusing on CI/process changes only (no product code impact).
Changes:
- Standardizes APIScan settings under
globalSdl(including registeredsoftwareName/softwareVersion) and removes redundant per-job configuration. - Updates Roslyn Analyzers execution to use
autoMsBuildInfowith a preceding “throwaway”dotnet buildto generate detectable build logs. - Modernizes Windows build environment by moving the Windows container image to
ltsc2025/vse2026and aligning the OneBranch Windows host version.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| eng/pipelines/onebranch/variables/onebranch-variables.yml | Updates OneBranch Windows container image to ltsc2025/vse2026. |
| eng/pipelines/onebranch/steps/roslyn-analyzers-buildproj-step.yml | Switches Roslyn Analyzers to auto-detect mode and adds a preceding build to generate logs. |
| eng/pipelines/onebranch/sqlclient-official.yml | Enables APIScan break in official, moves APIScan registration into globalSdl, and updates host/container notes. |
| eng/pipelines/onebranch/sqlclient-non-official.yml | Moves APIScan registration into globalSdl and adjusts SDL tool break/TSA settings (but currently conflicts with PR description). |
| eng/pipelines/onebranch/jobs/publish-symbols-job.yml | Removes redundant BinSkim break variable in a job where BinSkim is disabled. |
| eng/pipelines/onebranch/jobs/build-buildproj-job.yml | Removes per-job APIScan software identity variables, keeping only folder paths; drops unused packageVersion passthrough to Roslyn template. |
926d95a to
6ba04d4
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (3)
eng/pipelines/onebranch/sqlclient-official.yml:160
- The comment references the per-job variable name as
ob_sdl_apiscan_enable, but the actual OneBranch variable used elsewhere in this repo isob_sdl_apiscan_enabled(for example injobs/publish-symbols-job.yml). This is likely to mislead future edits/troubleshooting.
# We want APIScan enabled by default for our jobs. However, some jobs don't produce any
# artifacts, and they will disable APIScan via the OneBranch per-job ob_sdl_apiscan_enable
# variable.
eng/pipelines/onebranch/sqlclient-non-official.yml:139
- The comment references the per-job variable name as
ob_sdl_apiscan_enable, but the actual OneBranch variable used elsewhere in this repo isob_sdl_apiscan_enabled(seejobs/publish-symbols-job.yml). This mismatch can confuse maintainers when they try to disable APIScan for specific jobs.
# We want APIScan enabled by default for our jobs. However, some jobs don't produce any
# artifacts, and they will disable APIScan via the OneBranch per-job ob_sdl_apiscan_enable
# variable.
eng/pipelines/onebranch/sqlclient-non-official.yml:147
- PR description says the non-official pipeline now has SDL tools set to
break: true(and TSA publishing enabled), but this YAML setsbreak: falsefor APIScan/Armory/BinSkim/Policheck/Roslyn and keepstsa.enabled: false. Please align the PR description with the actual behavior, or adjust these settings to match the intended policy.
# APIScan errors should not break non-official builds.
break: false
# Use pre-release mode for non-official pipelines.
modeType: prerelease
- Fix typo in the APIScan enable comment: ob_sdl_apiscan_enable -> ob_sdl_apiscan_enabled (the actual per-job variable name), in both the official and non-official pipelines. - Drop "and version" from the non-official softwareName comment so it matches the official wording (the version has its own comment below).
mdaigle
left a comment
There was a problem hiding this comment.
Just the one question. Otherwise looks straightforward.
…pelines - Configure APIScan registration (softwareName, softwareVersion, modeType, verbosityLevel) inline in both official and non-official globalSdl blocks instead of per-job ob_sdl_apiscan_* variables; keep only DLL/PDB folders per-job in build-buildproj-job. - Register the actual SqlClient APIScan software version (6.10). - Enable break: true for apiscan, armory, binskim, policheck, and roslyn in the official pipeline; make the non-official pipeline non-breaking and disable its TSA publishing. - Drop redundant ob_sdl_binskim_break in onebranch-variables and publish-symbols-job (BinSkim disabled there).
- Fix typo in the APIScan enable comment: ob_sdl_apiscan_enable -> ob_sdl_apiscan_enabled (the actual per-job variable name), in both the official and non-official pipelines. - Drop "and version" from the non-official softwareName comment so it matches the official wording (the version has its own comment below).
- Drive every SDL tool's break value from the breakOnSdlError parameter and set tsa.enabled to its negation, instead of relying on OneBranch's implicit TSA-based break defaults (which have been buggy in the past). - Fix CodeQL config: nest enabled under compiled (codeql.compiled.enabled). - Add eslint break preemptively (kept disabled). - Add psscriptanalyzer for PowerShell script analysis. - Document the auto-injected SDL tasks that run without explicit config.
0549f32 to
3ea9d05
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (1)
eng/pipelines/onebranch/sqlclient-non-official.yml:256
- PR description says the non-official pipeline disables TSA publishing, but
globalSdl.tsa.enabledis currently driven bynot(breakOnSdlError)(defaultbreakOnSdlError: false), which enables TSA by default. This should be aligned with the stated policy (or the PR description updated).
tsa:
# We deliberately drive every tool's `break` value from breakOnSdlError and set TSA's
# `enabled` to its negation, rather than relying on OneBranch's implicit behaviour of
# flipping each tool's break default based on whether TSA is enabled. We have hit bugs in
# that implicit break-altering logic in the past, so we set both values explicitly to keep
# them deterministic.
#
# TSA bug filing is enabled when we are not breaking on SDL errors.
enabled: ${{ not(parameters.breakOnSdlError) }}
configFile: '$(REPO_ROOT)/.config/tsaoptions.json'
Add .config/guardian/.gdnbaselines with 9 pre-existing suppressible SDL findings (PoliCheck x2, PSScriptAnalyzer x7) generated by Guardian. Point globalSdl.baseline.baselineFile at it in both the non-official and official OneBranch pipelines so break=true fails only on new findings. Validated against build 162654: all 9 baselined signatures still present, no new suppressible findings, no stale entries. Remaining APIScan documentationnotfound/unpublisheddependency are notSuppressible and are handled via APIScan SEL whitelist requests.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
eng/pipelines/onebranch/sqlclient-official.yml:11
- Repo OneBranch guidance says the official pipeline should CI-trigger on
internal/mainin addition to the daily schedule (.github/instructions/onebranch-pipeline-design.instructions.md:12). Withtrigger: none, the pipeline will no longer run on branch activity. Please either restore theinternal/maintrigger (keepingpr: none) or update the repo guidance if the policy changed.
# No activity-based triggers.
pr: none
trigger: none
eng/pipelines/onebranch/sqlclient-non-official.yml:17
- The repo’s OneBranch pipeline guidance documents
sqlclient-non-official.ymlas “manual only” (no automated triggers) (.github/instructions/onebranch-pipeline-design.instructions.md:13). This PR adds a daily schedule oninternal/main, which changes that behavior. Please reconcile by either removing the schedule or updating the repo guidance to reflect the new automation policy.
# We run this pipeline on a daily schedule.
schedules:
- cron: "30 4 * * *"
displayName: Daily 04:30 UTC Build
branches:
eng/pipelines/onebranch/sqlclient-non-official.yml:262
- PR description says the non-official pipeline “disabled its TSA publishing”, but this config enables TSA bug filing whenever
breakOnSdlErroris false (the default):enabled: ${{ not(parameters.breakOnSdlError) }}. Please update the PR description to match the new conditional TSA behavior (or change the default/policy if TSA should stay off for non-official runs).
enabled: ${{ not(parameters.breakOnSdlError) }}
Hardcode TSA bug filing enabled in the official pipeline and disabled in the non-official pipeline, making it independent of breakOnSdlError. The non-official pipeline is a manual-only developer-helper build, so it should not file/update TSA bugs; bug filing is owned by the official pipeline. Also for non-official: remove the daily schedule (manual-only) and change the breakOnSdlError default to true. Expand the tsa block docs in both pipelines to explain how TSA flips each SDL tool's break default (which we override explicitly via breakOnSdlError) and that it is otherwise independent of breakOnSdlError.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
eng/pipelines/onebranch/sqlclient-official.yml:11
trigger: noneremoves the CI trigger forinternal/main, so the official pipeline will no longer run on pushes to that branch (only on the daily schedule). This also deviates from the repo’s OneBranch pipeline guidelines, which state the official pipeline should have a CI trigger oninternal/mainin addition to the schedule (.github/instructions/onebranch-pipeline-design.instructions.md:12). If this change is intentional, the guideline (and possibly the PR description) should be updated; otherwise, restore the branch trigger block.
# No activity-based triggers.
pr: none
trigger: none
…line-design instructions - sqlclient-official.yml: reword breakOnSdlError parameter description; TSA bug filing is hardcoded enabled and independent of breakOnSdlError (was incorrectly documented as toggling with it). - onebranch-pipeline-design.instructions.md: document the official pipeline as daily-schedule-only with no activity-based trigger (trigger: none), matching the current pipeline configuration. Resolves copilot-pull-request-reviewer threads r3640035708 and r3637718905.
| # Similar to the software name, we have a single version registered as well. This has | ||
| # nothing to do with the NuGet package version. It is purely an APIScan registration | ||
| # value that points to our backend configuration. | ||
| softwareVersion: '6.10' |
There was a problem hiding this comment.
It seems like we should 'ideally' be registering a new version every release, can we register 7.1.0 for this cycle and make it a practice to update and re-register after the release it out?
Will doing so trigger the same APIScan/TSA errors everytime we register a new version? What does the APIScan support say about it?
There was a problem hiding this comment.
Yes, I discussed this yesterday during APIScan office hours, and opened AB#46589. We will update the OneBranch pipelines after we get the new APIScan registrations completed. This PR will fix the builds in the meantime.
Codecov Report✅ All modified and coverable lines are covered by tests.
Additional details and impacted files@@ Coverage Diff @@
## main #4462 +/- ##
==========================================
- Coverage 70.88% 62.29% -8.59%
==========================================
Files 288 283 -5
Lines 43935 66821 +22886
==========================================
+ Hits 31142 41628 +10486
- Misses 12793 25193 +12400
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mdaigle
left a comment
There was a problem hiding this comment.
What matters most to me is that we're quickly made aware of any issues (which we'll get via scheduled build failures and TSA bugs), that we don't release with new issues (which we get via break: true in official), and that we can easily remediate issues and test them (which we get via non-official as breaking). So LGTM.
Description
Standardizes and modernizes the OneBranch SDL / APIScan configuration across the official and non-official pipelines. Engineering-process/CI only — no product code or public API is affected.
APIScan registration
softwareName,softwareVersion,modeType,verbosityLevel) inline into theglobalSdlblock of both pipelines, replacing the per-jobob_sdl_apiscan_*variables. Only the per-job DLL/PDB folder paths remain inbuild-buildproj-job.6.10).SDL break policy —
breakOnSdlErrorparameterbreakOnSdlErrorpipeline parameter (defaulttruein both pipelines) that drives thebreakvalue of every SDL tool: apiscan, armory, binskim, eslint, policheck, psscriptanalyzer, and roslyn.breakexplicitly avoids relying on OneBranch's implicit "TSA-enabled flips break defaults" behavior, which we've hit bugs with before.codeql.compiled.enabled: true).ob_sdl_binskim_breakvariable inonebranch-variablesandpublish-symbols-job(BinSkim is disabled there).TSA bug filing (decoupled from
breakOnSdlError)breakOnSdlError.breakOnSdlError) instead of filing/duplicating bugs.Baseline
.config/guardian/.gdnbaselinessnapshotting the pre-existing suppressible findings (PoliCheck ×2, PSScriptAnalyzer ×7) so the pipelines only break on NEW findings while we burn down the existing ones (AB#46588). Referenced viaglobalSdl.baseline.baselineFilein both pipelines.Triggers
pr: none,trigger: none, daily schedule at 04:30 UTC oninternal/main.pr: none,trigger: none).onebranch-pipeline-design.instructions.mdto match.GOTCHA: With
breakOnSdlErrordefaulting totrue, the Official pipeline will BREAK on any SDL errors, forcing us to fix them before releases (a good thing IMO). APIScan errors arenotSuppressible, so Official will break until our outstanding APIScan SEL exceptions are approved.Testing
breakOnSdlError: true): 26204.1 — surfaced the 6 APIScan errors + PSScriptAnalyzer findings, confirming the break wiring works.breakOnSdlError: false): 26204.2 — all SDL scans completed and published their analysis artifacts; the run was later canceled by an unrelated agent-connectivity timeout in the AKV build job. Baseline diff against the produced findings confirmed the 9 baselined signatures fully cover the suppressible findings (no new or stale entries).