Fix high surrogate calculation for SMP characters in UTF::ToUTF16

[yxing-goog]

Fix high surrogate calculation for SMP characters in UTF::ToUTF16

Context

Unicode code points beyond the Basic Multilingual Plane (BMP, U+0000..U+FFFF) reside in the Supplementary Planes (SMP / SIP / TIP / SSP / PUA, U+10000..U+10FFFF). Because UTF-16 uses 16-bit code units, characters in the range U+10000..U+10FFFF must be encoded as a surrogate pair consisting of:

1. A High (Lead) Surrogate in the reserved range [0xD800, 0xDBFF]
2. A Low (Trail) Surrogate in the reserved range [0xDC00, 0xDFFF]

According to Section 3.9 (Unicode Encoding Forms) of the Unicode Standard, the transformation from a Unicode scalar value uni to UTF-16 surrogate code units is:

• U' = uni - 0x10000 (a 20-bit value in range 0x00000..0xFFFFF)
• High Surrogate = 0xD800 + (U' >> 10)
• Low Surrogate = 0xDC00 + (U' & 0x3FF)

Root Cause Analysis

In src/text/utf.cpp, UTF::ToUTF16 attempted an arithmetic optimization to compute the high surrogate using the bitwise expression:
utf16[0] = castTo<uint16_t>((0xD800 - 64) | (uni >> 10));

Here, 0xD800 - 64 evaluates to 0xD7C0 (0b1101_0111_1100_0000). The intended shortcut assumed that 0xD7C0 | (uni >> 10) would simulate subtracting 0x10000 (which is 0x40 or 64 when shifted right by 10) and adding 0xD800.

However, bitwise OR | is only equivalent to addition when the operands share no overlapping 1-bits. Because 0xD7C0 has multiple 1-bits set in its lower byte (e.g., bit 6 and bit 7), applying | (uni >> 10) fails to perform a true subtraction and carry propagation. For many Supplementary Multilingual Plane (SMP) characters, the resulting value is strictly less than 0xD800 and lands in the unassigned or non-surrogate Unicode block.

Example (Mathematical Bold Capital 'M', U+1D40C):

• Expected: U' = 0x1D40C - 0x10000 = 0xD40C High Surrogate = 0xD800 + (0xD40C >> 10) = 0xD800 + 0x35 = 0xD835 Low Surrogate = 0xDC00 | (0x1D40C & 0x3FF) = 0xDC00 | 0x00C = 0xDC0C Resulting Pair: [0xD835, 0xDC0C] (Valid UTF-16 for '𝐌')
• Previous Implementation: (0xD800 - 64) | (0x1D40C >> 10) = 0xD7C0 | 0x75 = 0xD7F5 Resulting Pair: [0xD7F5, 0xDC0C] 0xD7F5 is NOT a valid high surrogate (valid range is 0xD800..0xDBFF).

When passed to downstream text layout engines, CoreText/DirectWrite/HarfBuzz, or string bridging APIs:

1. The invalid lead code unit 0xD7F5 fails surrogate pair decoding.
2. Shapers fail font fallback resolution, resulting in missing glyphs, replacement boxes ('B D B D' or [?]), or assertion crashes during text measurement and run layout.

Solution

Replace the bitwise approximation with the standard, canonical Unicode arithmetic:
utf16[0] = castTo<uint16_t>(0xD800 + ((uni - 0x10000) >> 10));

This ensures mathematical precision and strict bounds compliance [0xD800, 0xDBFF] for all 1,048,576 supplementary code points (U+10000..U+10FFFF).

Verification

• Tested boundary code points (U+10000, U+1D400..U+1D7FF, U+1F600, U+10FFFF).
• Verified valid UTF-16 round-trip decoding with system shaper fallback on iOS/macOS and data binding with non-BMP character strings (e.g. "𝐌𝐔𝐍𝐃𝐈 𝐎𝐏𝐔𝐒").