Skip to content

Fix memory leak in caching execution information location, fixes #8623 - #8625

Draft
mattcasters wants to merge 3 commits into
apache:mainfrom
mattcasters:issue-8623
Draft

mattcasters wants to merge 3 commits into
apache:mainfrom
mattcasters:issue-8623

Conversation

@mattcasters

Copy link
Copy Markdown
Contributor

Fixes #8623

Description

This PR addresses and fixes the memory leak observed when using the caching database (and file) execution information locations:

  1. Bounded In-Memory LRU Cache:

    • Replaced unbounded HashMap in BaseCachingExecutionInfoLocation with an access-order LinkedHashMap.
    • Added maxCacheSize metadata property and GUI widget (default: 50, order "905"), configurable in the metadata dialog.
    • Reduced maxCacheAge default from 24 hours to 10 minutes (600000 ms) and updated i18n tooltips and labels.
    • Implemented enforceMaxCacheSize() which evicts LRU entries and persists dirty entries prior to eviction.
    • Ensured close() clears the cache (finally { cache.clear(); }) and synchronized clearCaches().
  2. Eliminated Immortal Unwritten Cache Entries:

    • In CacheEntry.isTooOld(), fixed the condition so entries fall back to creationDate when lastRead and lastWritten are null, preventing entries from lingering indefinitely.
  3. Prevented PreparedStatement Collision & Statement Leak in retrieveIds():

    • In CachingDatabaseExecutionInfoLocation.retrieveIds(), collected parent IDs into a list first and closed the outer query/ResultSet before calling loadCacheEntry() for child executions, preventing statement overwrite and leak on Database.pstmt.
    • Reused a thread-safe Jackson ObjectMapper instance with HopJson.newMapper().
  4. Resource Safety in Core Database:

    • In Database.insertRow(schemaName, tableName, fields, data), wrapped insert execution in try ... finally { closeInsert(); } to ensure prepared statements are always closed even on error.
  5. Lifecycle and Execution Info Timer Hardening:

    • In Pipeline.fireExecutionFinishedListeners(), ensured pipelineCompleted(), extension point, and releaseVfsNamespace() are guaranteed to execute even if a listener throws an exception.
    • In LocalPipelineEngine, made stopTransformExecutionInfoTimer() synchronized and idempotent, stopped the timer on startup failure in startThreads(), and registered an execution-stopped listener to trigger timer/location cleanup.
  6. Tests:

    • Added unit tests in CachingDatabaseExecutionInfoLocationTest:
      • lruCacheEvictionEnforcesMaxSize: verifies LRU eviction bounds memory at maxCacheSize while evicted entries remain persistent in the database.
      • closeClearsCacheMap: verifies memory cache is cleared on close().
      • retrieveIdsWithChildrenLoadsChildrenCorrectly: verifies child entry retrieval works without statement collisions.

  • Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

…he#8623

- Bound in-memory cache to 50 LRU entries by default with LinkedHashMap
- Reduce default maxCacheAge to 10 minutes (600000 ms)
- Add maxCacheSize and update maxCacheAge GUI widgets and i18n
- Enforce LRU eviction in BaseCachingExecutionInfoLocation and clear cache on close()
- Fix CacheEntry.isTooOld() to eliminate immortal unwritten entries
- Fix query and PreparedStatement collision/leak in CachingDatabaseExecutionInfoLocation.retrieveIds()
- Reuse Jackson ObjectMapper with HopJson.newMapper()
- Protect Database.insertRow() with try-finally closeInsert()
- Harden Pipeline and LocalPipelineEngine lifecycle so timers and locations are always closed on completion, abort, or error
… write

The live pipeline entry copied the whole log buffer on every tick. Store
only the new lines and keep the newest 2 million characters. Leave an
entry in memory when its save fails, and do not close the location on a
safe stop. A missing max cache age stays at one day; new locations use
10 minutes.
…n info

A single-threaded parent drives the mapping again on every iteration, and
the child does not finish between batches. A location on that child kept a
second caching session open for the whole parent run. Drop the location
from a copy of the run configuration, and stop the child on dispose.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Memory leak while processing kafka topic

1 participant