Skip to content

[MINOR] OOC Fix Blocking Threads#2489

Merged
janniklinde merged 1 commit into
apache:mainfrom
janniklinde:ooc/fix-blocking-threadpools
Jun 15, 2026
Merged

[MINOR] OOC Fix Blocking Threads#2489
janniklinde merged 1 commit into
apache:mainfrom
janniklinde:ooc/fix-blocking-threadpools

Conversation

@janniklinde

Copy link
Copy Markdown
Contributor

Blocking threads now use a dynamic thread pool.

@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.40%. Comparing base (17803b1) to head (5542583).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2489      +/-   ##
============================================
- Coverage     71.41%   71.40%   -0.01%     
+ Complexity    48817    48815       -2     
============================================
  Files          1572     1572              
  Lines        189089   189090       +1     
  Branches      37101    37101              
============================================
- Hits         135030   135014      -16     
- Misses        43596    43612      +16     
- Partials      10463    10464       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@janniklinde janniklinde merged commit 3b564bb into apache:main Jun 15, 2026
50 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in SystemDS PR Queue Jun 15, 2026

protected CompletableFuture<Void> submitOOCTask(Runnable r, StreamContext ctx) {
ExecutorService pool = CommonThreadPool.get();
ExecutorService pool = CommonThreadPool.getDynamicPool();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not make a difference, The CommonThreadPool.get() should be the right way to get our shared threadpool that does get killed with the Java process.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should make a difference because submitOOCTask may be used for blocking work (e.g. dequeue()). If we now use a fixed sized thread pool (which especially causes problems in environments with fewer enabled CPUs thus smaller number of workers on the default thread pool), this may block scheduled tasks from making progress that are needed for those workers to unblock.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay did not know you had more tasks than threads in the pool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants