Support per-channel weights end to end

[mcremon-meta]

Summary:
Per-channel weight quantization went through prepare_pt2/convert_pt2
and then quietly stopped being quantized: fusion bailed on the
per-channel dequant, so the convs fell out to float cadence.fully_connected.
On HiFi it is worse than a fallback, because op_quantized_linear_out
reads out_multiplier[0] even in the tensor-qparam overload and would
requantize every channel with channel 0's scale, silently.

This makes the whole path work, AoT and kernels. Granularity is carried by
the existing overload split, with .out (tensor qparams) now being the
per-channel form and .per_tensor_out the per-tensor one. No new operator
names are introduced.

Only the generic kernels implement per-channel, so operator_fallback.bzl
routes every per-channel op there: the HiFi and TIE rows for the four
tensor-qparam ops are removed, and quantized_depthwise_conv1d_{ncl,nlc}.out
gain generic rows they never had. Backends fall back instead of failing, and
a follow-up can reclaim performance by adding a row back. NNLib's conv entry
points are already per-channel capable, so HiFi conv is the obvious first
candidate; its fully-connected and matmul entry points take a single
(multiplier, shift) pair and structurally cannot be.

The two new depthwise rows also need schemas in
min_runtime/custom_ops.yaml: that file is not OSS-only, the Jarvis xtensa
codegen resolves every selective-build op name through it and fails the
build outright on one it cannot find.

• The quantizer aligns a pattern's derived bias spec with the granularity
  of its weight spec. Previously six patterns hardcoded per_tensor_affine
  on the bias, so bias_scale = act_scale * weight_scale produced a vector
  that a per-tensor spec could not hold (ValueError: only one element tensors can be converted to Python scalars). CadenceAtenQuantizer.annotate
  is the one place that sees both specs, so the alignment lives there.

• Fusion accepts a per-channel weight dequant and emits the tensor-qparam
  overload, materializing bias_scale / out_multiplier / out_shift /
  weight_zero_point as lifted constants. Reading the weight scale and
  adding constants both need the ExportedProgram, which PassBase.call
  does not get, so QuantFusionPass takes one and stashes it for the
  duration of the pass rather than changing fuse() on ~20 patterns.

• Both conv and linear fuse per-channel. Linear is what fully-connected
  models are made of, so leaving it out would have made this a no-op for
  the target model. fuse_linear mirrors fuse_conv; quantized_linear
  has no bias_scale operand, so the scale reaches the kernel only through
  out_multiplier/out_shift. AddmmPattern still declines: its weight is
  [in_features, out_features], so an output-channel axis of 0 is
  meaningless there, and linear is preserved from decomposition anyway.

• Depthwise selection now runs before the per-channel overload swap in
  fuse_conv. It compared against the .per_tensor target after the swap
  had already moved to .default, so a per-channel depthwise conv silently
  stayed a dense conv. quantized_depthwise_conv1d_ncl/_nlc had no
  .default overload at all; the schemas, metas and reference
  implementations are added here.

• The conv lowering exits carry per-channel through.
  ReplaceConvWithChannelLastConvPass picks the layout from the input rank
  as before but takes the overload from the incoming node, instead of
  hardcoding .per_tensor. ReplaceConvWithIm2RowAndLinear normally
  recomputes out_multiplier/out_shift from bias_scale / out_scale, which
  is scalar arithmetic; for per-channel it reuses the conv's existing
  multiplier/shift nodes, which were derived from exactly that ratio at
  fusion time.

• The generic conv kernels do per-channel requantization. The conv2d and
  conv1d cores take weight_zero_point and bias_scale as a pointer plus a
  stride, resolved once per output channel: stride 0 keeps every channel on
  element 0 (per-tensor), stride 1 walks the channel axis. Existing callers
  pass 0, so per-tensor behaviour and the vectorizable inner loops are
  unchanged. The .out entry points previously read element 0 and applied
  channel 0's scale to every output channel.

• The conv reference implementations do real per-channel requantization.
  The four tensor-qparam variants used to call .item() on their qparams
  and delegate to the per-tensor path, which raised on a vector.
  quantized_conv_per_tensor becomes quantized_conv_common and broadcasts
  weight_zero_point and bias_scale over the output-channel axis.

Differential Revision: D117164581

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22227

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 5 Unrelated Failures

As of commit 9245a68 with merge base 35adf60 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  examples/models/test/test_export.py::ExportTest::test_efficient_sam_export_to_executorch
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 9dd036b9eb92537c0c39d58d66015fe9c50dce82a4811079f276413340f2053d /exec failed with exit code 1

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh) (matched linux rule in flaky-rules.json)
  The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
• pull / unittest-editable / macos / macos-job (gh) (similar failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest / windows / windows-job (gh) (trunk failure)
• pull / unittest-editable / windows / windows-job (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@mcremon-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D117164581.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.