Only claim conv overloads the QAT fold can actually match - #22241
Conversation
Summary: `annotate_conv_bn_partitions` listed `aten.conv2d.padding` and `aten.convolution.default` in `CONV_TARGETS`, but torchao can never fold either. `_fuse_conv_bn_qat` and `_fold_conv_bn_qat` both trace their patterns from `F.conv1d`, `F.conv2d`, `F.conv_transpose1d` and `F.conv_transpose2d`, so only those four overloads can match. Claiming a conv the fold cannot match is worse than not claiming it. The pass drops the conv's output qspec on the assumption the fold will move the bn output observer onto the conv; when the fold misses, the conv is left emitting float32 into a quantized graph and QNN rejects it with `0x232 != 0x408` (FLOAT_32 meeting UFIXED_POINT_8) -- the very failure the module docstring describes. `nn.Conv2d(..., padding="same")` exports to `aten.conv2d.padding` and hit exactly this. `aten.convolution.default` only appears after decomposition, long after this pass runs, so it never matched anything. Measured on master, conv+bn+relu through `prepare_qat_pt2e` -> `convert_pt2e` with `QnnQuantizer(8a8w, is_qat=True)`: | model | exported target | in CONV_TARGETS | bn survives | conv output unquantized | | --- | --- | --- | --- | --- | | `Conv2d(padding=1)` | `conv2d.default` | yes | no | 0 | | `Conv2d(padding="same")` | `conv2d.padding` | **yes** | yes | **1** | | `Conv1d(padding="same")` | `conv1d.padding` | no | yes | 0 | | `ConvTranspose2d` | `conv_transpose2d.input` | yes | no | 0 | `conv1d.padding` is the control: it also fails to fold -- that is a torchao limitation this pass cannot fix -- but because it was never claimed, the per-node annotator still gives the conv an output qspec and the graph stays valid. Only the overload this pass claimed lost its output. After narrowing `CONV_TARGETS`, the `conv2d.padding` case reports 0 unquantized conv outputs. Note `_is_conv_or_conv_transpose_node` is not a usable guard here -- it returns True for `conv2d.padding` (`torchao/quantization/pt2e/utils.py`, `_is_conv_node`). Differential Revision: D117778197
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22241
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 7 Pending, 1 Unrelated FailureAs of commit d4e9d75 with merge base d0c2da3 ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@billmguo has exported this pull request. If you are a Meta employee, you can view the originating Diff in D117778197. |
This PR needs a
|
qti-horodnic
left a comment
There was a problem hiding this comment.
LGTM, thanks for fixing the issue quickly
Summary:
annotate_conv_bn_partitionslistedaten.conv2d.paddingandaten.convolution.defaultinCONV_TARGETS, but torchao can never fold either._fuse_conv_bn_qatand_fold_conv_bn_qatboth trace their patterns fromF.conv1d,F.conv2d,F.conv_transpose1dandF.conv_transpose2d, so onlythose four overloads can match.
Claiming a conv the fold cannot match is worse than not claiming it. The pass
drops the conv's output qspec on the assumption the fold will move the bn output
observer onto the conv; when the fold misses, the conv is left emitting float32
into a quantized graph and QNN rejects it with
0x232 != 0x408(FLOAT_32 meetingUFIXED_POINT_8) -- the very failure the module docstring describes.
nn.Conv2d(..., padding="same")exports toaten.conv2d.paddingand hit exactlythis.
aten.convolution.defaultonly appears after decomposition, long afterthis pass runs, so it never matched anything.
Measured on master, conv+bn+relu through
prepare_qat_pt2e->convert_pt2ewith
QnnQuantizer(8a8w, is_qat=True):Conv2d(padding=1)conv2d.defaultConv2d(padding="same")conv2d.paddingConv1d(padding="same")conv1d.paddingConvTranspose2dconv_transpose2d.inputconv1d.paddingis the control: it also fails to fold -- that is a torchaolimitation this pass cannot fix -- but because it was never claimed, the per-node
annotator still gives the conv an output qspec and the graph stays valid. Only
the overload this pass claimed lost its output. After narrowing
CONV_TARGETS,the
conv2d.paddingcase reports 0 unquantized conv outputs.Note
_is_conv_or_conv_transpose_nodeis not a usable guard here -- it returnsTrue for
conv2d.padding(torchao/quantization/pt2e/utils.py,_is_conv_node).Differential Revision: D117778197