[doc](inverted-index) Document the norms index property and its VARIANT default

docs/sql-manual/basic-element/sql-data-types/semi-structured/VARIANT.md

@@ -1,4 +1,4 @@
 ---
 {
     "title": "VARIANT",
     "language": "en-US",
@@ -433,7 +433,7 @@
 In 3.1.x/4.0 and later, you can specify index properties for certain VARIANT subpaths, and even configure both tokenized and non-tokenized inverted indexes for the same path. Path-specific indexes require the path type to be declared via Schema Template.
 
 ```sql
--- Common properties: field_pattern (target path), analyzer, parser, support_phrase, etc.
+-- Common properties: field_pattern (target path), analyzer, parser, support_phrase, norms, etc.
 CREATE TABLE IF NOT EXISTS tbl (
     k BIGINT,
     v VARIANT<'content' : STRING>,
@@ -460,6 +460,23 @@
 SELECT * FROM tbl WHERE v['pattern_1'] = 'Doris';
 ```
 
+BM25 norms on VARIANT paths:
+
+A tokenized index on a VARIANT path writes BM25 norms just like one on an ordinary column. Norms store the record length used by BM25 and cost one byte per row for every indexed field, written even for rows that hold no value for that path. One segment keeps one index per path, so norms on a VARIANT column with many paths cost `rows × paths` bytes. Turning on the BE config `inverted_index_skip_norms_for_variant` (default `false`, changeable at runtime) drops them for every index on a VARIANT path, whatever that index's `norms` property says. Without norms, `MATCH` filtering still works, but a query that computes `score()` on that path returns an error, also while only some of the table's segments lack norms.
+
+Add `"norms" = "false"` to a path index that is never ranked with `score()`, and it stops paying for norms. An index without `field_pattern` passes the property to every subpath it covers:
+
+```sql
+CREATE TABLE IF NOT EXISTS tbl (
+    k BIGINT,
+    v VARIANT<'title' : STRING, 'body_*' : STRING>,
+    -- title keeps the default: norms, so it can be ranked with score()
+    INDEX idx_title(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "title"),
+    -- body_* is only filtered, never ranked: no norms, one byte less per row per path
+    INDEX idx_body(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "body_*", "norms" = "false")
+);
+```
+
 Note: 2.1.7+ supports only InvertedIndex V2 properties (fewer files, lower write IOPS; suitable for disaggregated storage/compute). 2.1.8+ removes offline Build Index.
 
 ### When indexes don’t work

docs/table-design/index/inverted-index/scoring.md

@@ -1,4 +1,4 @@
 ---
 {
     "title": "Relevance Scoring",
     "language": "en",
@@ -111,6 +111,32 @@
 
 When a query contains multiple terms, **the final score is the sum of the scores of each term**.
 
+### Record Length and the `norms` Property
+
+`|d|` comes from norms, which an index stores as one byte per row for every indexed field. The byte is written for every row of the segment, including rows that hold no value for that field, so an index that covers many sparse fields also pays for the rows it never matches.
+
+Norms are controlled per index with the `norms` property, which defaults to `true`:
+
+| Index                                  | Norms written                                                    |
+| -------------------------------------- | ---------------------------------------------------------------- |
+| Tokenized index on an ordinary column  | Yes, unless the index sets `"norms" = "false"`                   |
+| Tokenized index on a VARIANT path      | The same, unless the BE config below turns them off              |
+| Non-tokenized index                    | Never                                                            |
+
+A VARIANT path index is an index declared with `field_pattern`, together with the copy of it that each extracted subpath inherits. One segment holds one such index per path, so writing norms there costs `rows × paths` bytes. Turning on the BE config `inverted_index_skip_norms_for_variant` (default `false`, changeable at runtime) drops norms for every index on a VARIANT path, whatever that index's `norms` property says, so a cluster can reclaim that space without rewriting its index definitions.
+
+The `norms` property decides per index, and the copy inherited by a subpath carries the property of the index it comes from:
+
+```sql
+-- drop record-length normalization for one VARIANT path
+INDEX idx_body(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "body_*", "norms" = "false")
+
+-- drop it for an ordinary column
+INDEX idx_content(content) USING INVERTED PROPERTIES("parser" = "english", "norms" = "false")
+```
+
+Relevance scoring needs norms: a query that computes `score()` on a tokenized index returns an error when any segment it reads has no norms for that field, which includes a table where only some segments were written without them. `MATCH_*` filtering is not affected. Set `"norms" = "false"`, or turn the config on, only for indexes that are never ranked with `score()`. The property and the config apply to newly written segments; segments written earlier keep their norms until compaction rewrites them. Change either one only after every BE has been upgraded to a version that supports them.
+
 ## Interpreting the Results
 
 Understanding the scoring results helps you use relevance ranking more accurately:

i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/basic-element/sql-data-types/semi-structured/VARIANT.md

@@ -1,4 +1,4 @@
 ---
 {
     "title": "VARIANT",
     "language": "zh-CN",
@@ -433,7 +433,7 @@
 在 3.1.x/4.0 及之后的版本中，可为 VARIANT 的部分子列单独指定索引属性，甚至在同一路径上同时配置“分词与不分词”的两种倒排索引。指定 Path 索引需配合 Path 类型（Schema Template）使用。
 
 ```sql
--- 常用属性：field_pattern（目标子路径）、analyzer、parser、support_phrase 等
+-- 常用属性：field_pattern（目标子路径）、analyzer、parser、support_phrase、norms 等
 CREATE TABLE IF NOT EXISTS tbl (
     k BIGINT,
     v VARIANT<'content' : STRING>,
@@ -460,6 +460,23 @@
 SELECT * FROM tbl WHERE v['pattern_1'] = 'Doris';
 ```
 
+VARIANT 路径上的 BM25 norms：
+
+VARIANT 路径上的分词索引和普通列上的一样会写 BM25 norms。norms 保存 BM25 使用的记录长度，按行存储，每个被索引的字段每行 1 字节，即使该行在这个路径上没有值也会写入。一个段中每个路径各有一份索引，因此在路径数很多的 VARIANT 列上写 norms 的代价是 `行数 × 路径数` 字节。打开 BE 配置项 `inverted_index_skip_norms_for_variant`（默认 `false`，可动态修改）之后，VARIANT 路径上的所有索引都不再写 norms，无论该索引的 `norms` 属性如何。没有 norms 时 `MATCH` 过滤照常可用，但在该路径上计算 `score()` 的查询会报错；表中只有部分段缺少 norms 时也一样。
+
+如果某个路径从不用 `score()` 排序，给它的索引加上 `"norms" = "false"` 即可省下这部分空间。不带 `field_pattern` 的索引会把该属性传给自己覆盖的每个子路径：
+
+```sql
+CREATE TABLE IF NOT EXISTS tbl (
+    k BIGINT,
+    v VARIANT<'title' : STRING, 'body_*' : STRING>,
+    -- title 保持默认：写 norms，可以用 score() 排序
+    INDEX idx_title(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "title"),
+    -- body_* 只做过滤、不排序：不写 norms，每行每路径省 1 字节
+    INDEX idx_body(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "body_*", "norms" = "false")
+);
+```
+
 注意：2.1.7+ 仅支持 InvertedIndex V2 属性（文件更少、写入 IOPS 更低，适配存算分离）。2.1.8+ 不再支持离线 Build Index 构建。
 
 ### 索引失效问题
@@ -720,7 +737,7 @@
 
 方案二：扩展 `DESC` 展示已完成子列列式提取（Subcolumnization）的子路径：
 
 ```sql
 SET describe_extend_variant_column = true;
 DESC variant_tbl;
 ```

i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/index/inverted-index/scoring.md

@@ -1,4 +1,4 @@
 ---
 {
     "title": "相关性打分",
     "language": "zh",
@@ -111,6 +111,32 @@
 
 当查询包含多个词项时，**最终得分为各词项得分之和**。
 
+### 记录长度与 `norms` 属性
+
+公式中的 `|d|` 来自 norms。索引会为每个被索引的字段按行存储 norms，每行 1 字节。这个字节对段内每一行都会写入，包括该字段没有值的行，因此覆盖大量稀疏字段的索引也要为从不命中的行付出空间。
+
+norms 由索引属性 `norms` 控制，该属性默认为 `true`：
+
+| 索引                     | 是否写 norms                                             |
+| ------------------------ | -------------------------------------------------------- |
+| 普通列上的分词索引       | 写，除非索引设置了 `"norms" = "false"`                   |
+| VARIANT 路径上的分词索引 | 同上，除非下面的 BE 配置项把它们关掉                     |
+| 非分词索引               | 从不写                                                   |
+
+VARIANT 路径索引指用 `field_pattern` 声明的索引，以及每个被提取的子路径继承到的那份副本。一个段中每个路径各有一份这样的索引，因此在这里写 norms 的代价是 `行数 × 路径数` 字节。打开 BE 配置项 `inverted_index_skip_norms_for_variant`（默认 `false`，可动态修改）之后，VARIANT 路径上的所有索引都不再写 norms，无论该索引的 `norms` 属性如何，这样集群不必改写索引定义就能收回这部分空间。
+
+`norms` 属性按索引生效；子路径继承到的那份副本会带上它所来源的索引的属性：
+
+```sql
+-- 关闭某个 VARIANT 路径上的记录长度归一化
+INDEX idx_body(v) USING INVERTED PROPERTIES("parser" = "english", "field_pattern" = "body_*", "norms" = "false")
+
+-- 关闭普通列上的记录长度归一化
+INDEX idx_content(content) USING INVERTED PROPERTIES("parser" = "english", "norms" = "false")
+```
+
+相关性打分依赖 norms：在分词索引上计算 `score()` 的查询，只要读到的任意一个段缺少该字段的 norms 就会报错，只有部分段缺少 norms 的表也一样。`MATCH_*` 过滤不受影响。请只对从不用 `score()` 排序的索引设置 `"norms" = "false"` 或打开该配置项。该属性和配置项只对新写入的段生效；此前写入的段会保留原有的 norms，直到被 compaction 重写。请在所有 BE 都升级到支持它们的版本之后再修改。
+
 ## 结果解读
 
 理解打分结果有助于更准确地使用相关性排序：