Skip to content

Fix stat widgets freezing permanently after a transient sensor error#1036

Open
axectly wants to merge 3 commits into
mathoudebine:mainfrom
axectly:pr/stat-thread-resilience
Open

Fix stat widgets freezing permanently after a transient sensor error#1036
axectly wants to merge 3 commits into
mathoudebine:mainfrom
axectly:pr/stat-thread-resilience

Conversation

@axectly

@axectly axectly commented Jul 25, 2026

Copy link
Copy Markdown

Problem

A stat widget can freeze permanently while the rest of the program keeps running fine. It happens at random, after minutes or hours, and nothing is written to log.log.

Three things combine:

1. A scheduled job that raises kills its thread for good. Each stat is scheduled in its own thread with its own sched.scheduler (library/scheduler.py). An exception raised by the job propagates out of scheduler.run(), the thread exits and that widget is never updated again. The traceback is invisible in the packaged Windows executable, because threading.excepthook writes to sys.stderr, which is None there.

2. ping3 returns None instead of a number. On a ping timeout, Ping.stats() raises at int(delay). The None is also stored in the line graph history, and DisplayLineGraph() calls math.isnan() on every stored value, which raises on None for as long as the value stays in the buffer.

3. int() is called on values that are math.nan. Sensors already use math.nan to report "no value" — LibreHardwareMonitor does it for CPU load and fan speed when it cannot read them — but the display helpers convert with int(), which raises ValueError. For display_themed_percent_radial_bar() and display_themed_percent_value() the conversion happens before the SHOW check inside the called function, so the stat crashes even when the widget is hidden. This looks like the cause of #718.

Observed on Windows 11 / Python 3.13, HW_SENSORS: AUTO (LHM), Wi-Fi connection, PING: 8.8.8.8.

Cause 2, during a short network outage:

[ERROR] Scheduled job 'PingStats' failed, skipping this cycle
Traceback (most recent call last):
  File "library\scheduler.py", line 195, in PingStats
    stats.Ping.stats()
  File "library\stats.py", line 936, in stats
    value=int(delay),
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

and then, on the following cycles, once the network was back:

  File "library\stats.py", line 946, in stats
    display_themed_line_graph(theme_data['LINE_GRAPH'], cls.last_values_ping)
  File "library\lcd\lcd_comm.py", line 458, in DisplayLineGraph
    if not math.isnan(value):
TypeError: must be real number, not NoneType

Cause 3, twice in one day, with RADIAL not even shown in the theme:

[ERROR] Scheduled job 'CPUPercentage' failed, skipping this cycle
Traceback (most recent call last):
  File "library\stats.py", line 284, in percentage
  File "library\stats.py", line 208, in display_themed_percent_radial_bar
ValueError: cannot convert float NaN to integer

(the [ERROR] Scheduled job ... lines come from this PR; without it the thread dies silently at the first traceback)

Changes

  • library/scheduler.py: wrap the scheduled call in try/except, log the failure and keep going — the next run is already scheduled at that point, so the stat just skips one cycle. Logging is rate-limited to one message per second per job, because QueueHandler runs every millisecond and a permanently failing job would otherwise flood log.log (which rotates at 1 MB).
  • library/stats.py: normalize a missing ping delay to math.nan and skip, for that cycle, the widgets that need a number. save_last_value() maps None to math.nan as well, so no sensor returning None can poison a graph history — math.nan is already this code's "no data" marker, see last_values_list().
  • library/stats.py: skip drawing in the display helpers when the value is None or nan, instead of letting int() raise.

The changes are complementary: the last two remove the two known ways a stat can raise, the first one makes sure that any remaining one only costs a cycle instead of the whole thread, and becomes visible in the log.

SystemExit and KeyboardInterrupt derive from BaseException and are not caught, so the clean shutdown path in main.py is unaffected.

Tested on a Turing 5" (rev C) with Windows 11 and Python 3.13.


🤖 Generated with Claude Code

axectly and others added 3 commits July 25, 2026 21:42
Every stat is scheduled in its own thread with its own sched.scheduler. An
exception raised by the job propagates out of scheduler.run(), so that thread
dies and its widget freezes on screen until the program is restarted.

Nothing is logged when this happens: the thread exception goes to sys.stderr,
which is None in the packaged windowed executable.

Catch exceptions around the job instead. The next run is already scheduled at
that point, so the stat simply skips one cycle and recovers. Failures are now
logged, rate-limited to one message per second per job because some jobs (e.g.
QueueHandler) run every millisecond.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ping3 returns None on timeout (and False for an unknown host), so Ping.stats
raised TypeError on int(delay) as soon as one ping timed out, and the None was
also stored in the line graph history. DisplayLineGraph calls math.isnan() on
every stored value, which raises TypeError on None, so subsequent redraws of
that graph failed too, for as long as the value stayed in the history buffer.

Combined with a scheduled job exception killing its thread, a single ping
timeout was enough to freeze the ping widget permanently.

Normalize a missing ping delay to math.nan, which is what this code already
uses as its "no data" marker (see last_values_list), and skip the widgets that
need a number for that cycle. Guard save_last_value too, so any sensor
returning None cannot poison a graph history.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sensors already use math.nan to report "no value" (LibreHardwareMonitor does it
for CPU load and fan speed when it cannot read them), but the display helpers
convert the value with int(), which raises ValueError on nan and kills the
thread of that stat, freezing its widgets until the program is restarted:

    File "library/stats.py", line 208, in display_themed_percent_radial_bar
    ValueError: cannot convert float NaN to integer

For the radial bar and the text value the conversion also happens before the
SHOW check inside the called function, so a hidden widget crashes the stat too.

Skip drawing when there is no value, as reported in mathoudebine#718.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant