Skip to content

Force UTF-8 output from yt-dlp to fix mojibake in redirected streams - #81

Open
Aimeast wants to merge 1 commit into
Bluegrams:masterfrom
Aimeast:fix/utf8-pipe-encoding
Open

Force UTF-8 output from yt-dlp to fix mojibake in redirected streams#81
Aimeast wants to merge 1 commit into
Bluegrams:masterfrom
Aimeast:fix/utf8-pipe-encoding

Conversation

@Aimeast

@Aimeast Aimeast commented Aug 8, 2026

Copy link
Copy Markdown

Problem

On Windows, yt-dlp output captured through the library is garbled (mojibake).
For example the YouTube bot-check error message

Sign in to confirm you're not a bot

is reported as

Sign in to confirm you��re not a bot

Root cause

yt-dlp is a Python program. When its stdout/stderr are redirected to a pipe,
Python picks their encoding from the system locale at process startup
(locale.getpreferredencoding() -> the ANSI code page, e.g. GBK/cp936 on
Chinese Windows) instead of always using UTF-8. The library always reads the
process output as UTF-8 (StandardOutputEncoding = Encoding.UTF8), so every
non-ASCII byte sequence was decoded into U+FFFD replacement characters.

Confirmed with yt-dlp.exe -v:

[debug] Encodings: locale cp936, fs utf-8, pref cp936, out gbk (No ANSI), error gbk (No ANSI), screen gbk (No ANSI)

Reading the pipes as GBK works on a GBK-locale machine but breaks on machines
with a different ANSI code page (e.g. cp1252) and is not future-proof once
yt-dlp ships builds that default to UTF-8 (Python 3.15+).

Fix

Append --encoding utf-8 to the yt-dlp arguments unless the caller already
set OptionSet.Encoding, which is respected and left untouched. This forces
yt-dlp to emit UTF-8 on both stdout and stderr regardless of the machine's
locale, matching the encoding the output is always read with.

The public API is unchanged - only the internal ConvertToArgs method in
YoutubeDLSharp/YoutubeDLProcess.cs was modified.

Verification

  • dotnet build passes with 0 errors.
  • Byte-level: without the fix, non-ASCII bytes in yt-dlp's output are GBK
    (B2 E2 CA D4 for 测试); with --encoding utf-8 they are UTF-8
    (E6 B5 8B E8 AF 95).
  • End-to-end: ran YoutubeDLProcess.RunAsync with a real yt-dlp.exe
    (2026.07.04, PyInstaller build) on a URL whose error output contains
    non-ASCII text - before the fix every non-ASCII char became U+FFFD;
    after the fix the output decodes correctly with zero U+FFFD.
  • Argument handling: default OptionSet produces --encoding utf-8;
    OptionSet.Encoding = "gbk" produces --encoding "gbk" (not overridden).

@Aimeast
Aimeast force-pushed the fix/utf8-pipe-encoding branch from 2ccc3e6 to 797f44c Compare August 10, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant