Skip to content

Standardize APIScan registration and SDL break policy in OneBranch pipelines#4462

Open
paulmedynski wants to merge 6 commits into
mainfrom
dev/paul/apiscan
Open

Standardize APIScan registration and SDL break policy in OneBranch pipelines#4462
paulmedynski wants to merge 6 commits into
mainfrom
dev/paul/apiscan

Conversation

@paulmedynski

@paulmedynski paulmedynski commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Description

Standardizes and modernizes the OneBranch SDL / APIScan configuration across the official and non-official pipelines. Engineering-process/CI only — no product code or public API is affected.

APIScan registration

  • Moved APIScan registration (softwareName, softwareVersion, modeType, verbosityLevel) inline into the globalSdl block of both pipelines, replacing the per-job ob_sdl_apiscan_* variables. Only the per-job DLL/PDB folder paths remain in build-buildproj-job.
  • Registered the actual SqlClient APIScan software version (6.10).

SDL break policy — breakOnSdlError parameter

  • Added a breakOnSdlError pipeline parameter (default true in both pipelines) that drives the break value of every SDL tool: apiscan, armory, binskim, eslint, policheck, psscriptanalyzer, and roslyn.
  • Setting each tool's break explicitly avoids relying on OneBranch's implicit "TSA-enabled flips break defaults" behavior, which we've hit bugs with before.
  • Added PSScriptAnalyzer (previously not run) to scan the repo's PowerShell scripts.
  • Fixed the CodeQL config nesting (codeql.compiled.enabled: true).
  • Dropped the redundant ob_sdl_binskim_break variable in onebranch-variables and publish-symbols-job (BinSkim is disabled there).

TSA bug filing (decoupled from breakOnSdlError)

  • Official: TSA hardcoded enabled — SDL findings are always filed as bug work items, independent of breakOnSdlError.
  • Non-official: TSA hardcoded disabled — it is a manual developer-helper pipeline, so findings surface directly in the build (gated by breakOnSdlError) instead of filing/duplicating bugs.

Baseline

  • Added .config/guardian/.gdnbaselines snapshotting the pre-existing suppressible findings (PoliCheck ×2, PSScriptAnalyzer ×7) so the pipelines only break on NEW findings while we burn down the existing ones (AB#46588). Referenced via globalSdl.baseline.baselineFile in both pipelines.

Triggers

  • Official: pr: none, trigger: none, daily schedule at 04:30 UTC on internal/main.
  • Non-official: manual only (pr: none, trigger: none).
  • Updated onebranch-pipeline-design.instructions.md to match.

GOTCHA: With breakOnSdlError defaulting to true, the Official pipeline will BREAK on any SDL errors, forcing us to fix them before releases (a good thing IMO). APIScan errors are notSuppressible, so Official will break until our outstanding APIScan SEL exceptions are approved.

Testing

  • sqlclient-non-official — SDL break on (breakOnSdlError: true): 26204.1 — surfaced the 6 APIScan errors + PSScriptAnalyzer findings, confirming the break wiring works.
  • sqlclient-non-official — SDL break off (breakOnSdlError: false): 26204.2 — all SDL scans completed and published their analysis artifacts; the run was later canceled by an unrelated agent-connectivity timeout in the AKV build job. Baseline diff against the produced findings confirmed the 9 baselined signatures fully cover the suppressible findings (no new or stale entries).
  • sqlclient-official — pending; can only run with approved PR provenance (after merge). Expected to break on APIScan until the SEL exceptions are approved.

Copilot AI review requested due to automatic review settings July 21, 2026 13:44
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Jul 21, 2026
@paulmedynski paulmedynski moved this from To triage to In progress in SqlClient Board Jul 21, 2026
@paulmedynski paulmedynski added the Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems. label Jul 21, 2026
@paulmedynski paulmedynski added this to the 7.1.0-preview3 milestone Jul 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Modernizes the OneBranch official and non-official pipelines’ SDL/APIScan configuration to be more standardized and compatible with newer Windows container images, focusing on CI/process changes only (no product code impact).

Changes:

  • Standardizes APIScan settings under globalSdl (including registered softwareName/softwareVersion) and removes redundant per-job configuration.
  • Updates Roslyn Analyzers execution to use autoMsBuildInfo with a preceding “throwaway” dotnet build to generate detectable build logs.
  • Modernizes Windows build environment by moving the Windows container image to ltsc2025/vse2026 and aligning the OneBranch Windows host version.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
eng/pipelines/onebranch/variables/onebranch-variables.yml Updates OneBranch Windows container image to ltsc2025/vse2026.
eng/pipelines/onebranch/steps/roslyn-analyzers-buildproj-step.yml Switches Roslyn Analyzers to auto-detect mode and adds a preceding build to generate logs.
eng/pipelines/onebranch/sqlclient-official.yml Enables APIScan break in official, moves APIScan registration into globalSdl, and updates host/container notes.
eng/pipelines/onebranch/sqlclient-non-official.yml Moves APIScan registration into globalSdl and adjusts SDL tool break/TSA settings (but currently conflicts with PR description).
eng/pipelines/onebranch/jobs/publish-symbols-job.yml Removes redundant BinSkim break variable in a job where BinSkim is disabled.
eng/pipelines/onebranch/jobs/build-buildproj-job.yml Removes per-job APIScan software identity variables, keeping only folder paths; drops unused packageVersion passthrough to Roslyn template.

Comment thread eng/pipelines/onebranch/sqlclient-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml Outdated
Comment thread eng/pipelines/onebranch/steps/roslyn-analyzers-buildproj-step.yml Outdated
Comment thread eng/pipelines/onebranch/jobs/build-buildproj-job.yml
Comment thread eng/pipelines/onebranch/jobs/build-buildproj-job.yml
Comment thread eng/pipelines/onebranch/jobs/publish-symbols-job.yml
Comment thread eng/pipelines/onebranch/steps/roslyn-analyzers-buildproj-step.yml Outdated
Comment thread eng/pipelines/onebranch/steps/roslyn-analyzers-buildproj-step.yml Outdated
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (3)

eng/pipelines/onebranch/sqlclient-official.yml:160

  • The comment references the per-job variable name as ob_sdl_apiscan_enable, but the actual OneBranch variable used elsewhere in this repo is ob_sdl_apiscan_enabled (for example in jobs/publish-symbols-job.yml). This is likely to mislead future edits/troubleshooting.
        # We want APIScan enabled by default for our jobs.  However, some jobs don't produce any
        # artifacts, and they will disable APIScan via the OneBranch per-job ob_sdl_apiscan_enable
        # variable.

eng/pipelines/onebranch/sqlclient-non-official.yml:139

  • The comment references the per-job variable name as ob_sdl_apiscan_enable, but the actual OneBranch variable used elsewhere in this repo is ob_sdl_apiscan_enabled (see jobs/publish-symbols-job.yml). This mismatch can confuse maintainers when they try to disable APIScan for specific jobs.
        # We want APIScan enabled by default for our jobs.  However, some jobs don't produce any
        # artifacts, and they will disable APIScan via the OneBranch per-job ob_sdl_apiscan_enable
        # variable.

eng/pipelines/onebranch/sqlclient-non-official.yml:147

  • PR description says the non-official pipeline now has SDL tools set to break: true (and TSA publishing enabled), but this YAML sets break: false for APIScan/Armory/BinSkim/Policheck/Roslyn and keeps tsa.enabled: false. Please align the PR description with the actual behavior, or adjust these settings to match the intended policy.
        # APIScan errors should not break non-official builds.
        break: false

        # Use pre-release mode for non-official pipelines.
        modeType: prerelease

Comment thread eng/pipelines/onebranch/sqlclient-official.yml
@paulmedynski paulmedynski changed the title Modernize and standardize OneBranch SDL/APIScan pipeline configuration Standardize APIScan registration and SDL break policy in OneBranch pipelines Jul 21, 2026
paulmedynski added a commit that referenced this pull request Jul 21, 2026
- Fix typo in the APIScan enable comment: ob_sdl_apiscan_enable ->
  ob_sdl_apiscan_enabled (the actual per-job variable name), in both the
  official and non-official pipelines.
- Drop "and version" from the non-official softwareName comment so it
  matches the official wording (the version has its own comment below).
Copilot AI review requested due to automatic review settings July 21, 2026 17:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comment thread eng/pipelines/onebranch/variables/onebranch-variables.yml
@paulmedynski
paulmedynski marked this pull request as ready for review July 21, 2026 19:47
@paulmedynski
paulmedynski requested a review from a team as a code owner July 21, 2026 19:47
@paulmedynski
paulmedynski enabled auto-merge (squash) July 21, 2026 19:47
@paulmedynski paulmedynski moved this from In progress to In review in SqlClient Board Jul 21, 2026
Comment thread eng/pipelines/onebranch/sqlclient-official.yml Outdated

@mdaigle mdaigle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just the one question. Otherwise looks straightforward.

…pelines

- Configure APIScan registration (softwareName, softwareVersion, modeType,
  verbosityLevel) inline in both official and non-official globalSdl blocks
  instead of per-job ob_sdl_apiscan_* variables; keep only DLL/PDB folders
  per-job in build-buildproj-job.
- Register the actual SqlClient APIScan software version (6.10).
- Enable break: true for apiscan, armory, binskim, policheck, and roslyn in
  the official pipeline; make the non-official pipeline non-breaking and
  disable its TSA publishing.
- Drop redundant ob_sdl_binskim_break in onebranch-variables and
  publish-symbols-job (BinSkim disabled there).
- Fix typo in the APIScan enable comment: ob_sdl_apiscan_enable ->
  ob_sdl_apiscan_enabled (the actual per-job variable name), in both the
  official and non-official pipelines.
- Drop "and version" from the non-official softwareName comment so it
  matches the official wording (the version has its own comment below).
- Drive every SDL tool's break value from the breakOnSdlError parameter and
  set tsa.enabled to its negation, instead of relying on OneBranch's implicit
  TSA-based break defaults (which have been buggy in the past).
- Fix CodeQL config: nest enabled under compiled (codeql.compiled.enabled).
- Add eslint break preemptively (kept disabled).
- Add psscriptanalyzer for PowerShell script analysis.
- Document the auto-injected SDL tasks that run without explicit config.
Copilot AI review requested due to automatic review settings July 23, 2026 11:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

eng/pipelines/onebranch/sqlclient-non-official.yml:256

  • PR description says the non-official pipeline disables TSA publishing, but globalSdl.tsa.enabled is currently driven by not(breakOnSdlError) (default breakOnSdlError: false), which enables TSA by default. This should be aligned with the stated policy (or the PR description updated).
      tsa:
        # We deliberately drive every tool's `break` value from breakOnSdlError and set TSA's
        # `enabled` to its negation, rather than relying on OneBranch's implicit behaviour of
        # flipping each tool's break default based on whether TSA is enabled.  We have hit bugs in
        # that implicit break-altering logic in the past, so we set both values explicitly to keep
        # them deterministic.
        #
        # TSA bug filing is enabled when we are not breaking on SDL errors.
        enabled: ${{ not(parameters.breakOnSdlError) }}
        configFile: '$(REPO_ROOT)/.config/tsaoptions.json'

Comment thread eng/pipelines/onebranch/sqlclient-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml Outdated
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml Outdated
Copilot AI review requested due to automatic review settings July 23, 2026 16:29
Add .config/guardian/.gdnbaselines with 9 pre-existing suppressible SDL
findings (PoliCheck x2, PSScriptAnalyzer x7) generated by Guardian.
Point globalSdl.baseline.baselineFile at it in both the non-official and
official OneBranch pipelines so break=true fails only on new findings.

Validated against build 162654: all 9 baselined signatures still present,
no new suppressible findings, no stale entries. Remaining APIScan
documentationnotfound/unpublisheddependency are notSuppressible and are
handled via APIScan SEL whitelist requests.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/pipelines/onebranch/sqlclient-official.yml:11

  • Repo OneBranch guidance says the official pipeline should CI-trigger on internal/main in addition to the daily schedule (.github/instructions/onebranch-pipeline-design.instructions.md:12). With trigger: none, the pipeline will no longer run on branch activity. Please either restore the internal/main trigger (keeping pr: none) or update the repo guidance if the policy changed.
# No activity-based triggers.
pr: none
trigger: none

eng/pipelines/onebranch/sqlclient-non-official.yml:17

  • The repo’s OneBranch pipeline guidance documents sqlclient-non-official.yml as “manual only” (no automated triggers) (.github/instructions/onebranch-pipeline-design.instructions.md:13). This PR adds a daily schedule on internal/main, which changes that behavior. Please reconcile by either removing the schedule or updating the repo guidance to reflect the new automation policy.
# We run this pipeline on a daily schedule.
schedules:
  - cron: "30 4 * * *"
    displayName: Daily 04:30 UTC Build
    branches:

eng/pipelines/onebranch/sqlclient-non-official.yml:262

  • PR description says the non-official pipeline “disabled its TSA publishing”, but this config enables TSA bug filing whenever breakOnSdlError is false (the default): enabled: ${{ not(parameters.breakOnSdlError) }}. Please update the PR description to match the new conditional TSA behavior (or change the default/policy if TSA should stay off for non-official runs).
        enabled: ${{ not(parameters.breakOnSdlError) }}

Copilot AI review requested due to automatic review settings July 23, 2026 17:14
Hardcode TSA bug filing enabled in the official pipeline and disabled in
the non-official pipeline, making it independent of breakOnSdlError. The
non-official pipeline is a manual-only developer-helper build, so it should
not file/update TSA bugs; bug filing is owned by the official pipeline.

Also for non-official: remove the daily schedule (manual-only) and change
the breakOnSdlError default to true.

Expand the tsa block docs in both pipelines to explain how TSA flips each
SDL tool's break default (which we override explicitly via breakOnSdlError)
and that it is otherwise independent of breakOnSdlError.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

eng/pipelines/onebranch/sqlclient-official.yml:11

  • trigger: none removes the CI trigger for internal/main, so the official pipeline will no longer run on pushes to that branch (only on the daily schedule). This also deviates from the repo’s OneBranch pipeline guidelines, which state the official pipeline should have a CI trigger on internal/main in addition to the schedule (.github/instructions/onebranch-pipeline-design.instructions.md:12). If this change is intentional, the guideline (and possibly the PR description) should be updated; otherwise, restore the branch trigger block.
# No activity-based triggers.
pr: none
trigger: none

Comment thread eng/pipelines/onebranch/sqlclient-official.yml Outdated
Comment thread .config/guardian/.gdnbaselines
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-official.yml
Comment thread eng/pipelines/onebranch/sqlclient-official.yml
Copilot AI review requested due to automatic review settings July 23, 2026 17:30
…line-design instructions

- sqlclient-official.yml: reword breakOnSdlError parameter description; TSA
  bug filing is hardcoded enabled and independent of breakOnSdlError (was
  incorrectly documented as toggling with it).
- onebranch-pipeline-design.instructions.md: document the official pipeline
  as daily-schedule-only with no activity-based trigger (trigger: none),
  matching the current pipeline configuration.

Resolves copilot-pull-request-reviewer threads r3640035708 and r3637718905.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment thread eng/pipelines/onebranch/sqlclient-non-official.yml
@paulmedynski
paulmedynski requested a review from mdaigle July 23, 2026 17:47
# Similar to the software name, we have a single version registered as well. This has
# nothing to do with the NuGet package version. It is purely an APIScan registration
# value that points to our backend configuration.
softwareVersion: '6.10'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we should 'ideally' be registering a new version every release, can we register 7.1.0 for this cycle and make it a practice to update and re-register after the release it out?

Will doing so trigger the same APIScan/TSA errors everytime we register a new version? What does the APIScan support say about it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I discussed this yesterday during APIScan office hours, and opened AB#46589. We will update the OneBranch pipelines after we get the new APIScan registrations completed. This PR will fix the builds in the meantime.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.29%. Comparing base (d040700) to head (1fbca8d).

❗ There is a different number of reports uploaded between BASE (d040700) and HEAD (1fbca8d). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (d040700) HEAD (1fbca8d)
CI-SqlClient 1 0
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4462      +/-   ##
==========================================
- Coverage   70.88%   62.29%   -8.59%     
==========================================
  Files         288      283       -5     
  Lines       43935    66821   +22886     
==========================================
+ Hits        31142    41628   +10486     
- Misses      12793    25193   +12400     
Flag Coverage Δ
CI-SqlClient ?
PR-SqlClient-Project 62.29% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdaigle mdaigle left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What matters most to me is that we're quickly made aware of any issues (which we'll get via scheduled build failures and TSA bugs), that we don't release with new issues (which we get via break: true in official), and that we can easily remediate issues and test them (which we get via non-official as breaking). So LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Engineering Use this for issues that are targeted for changes in the 'eng' folder or build systems.

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

5 participants