Bug 5554: Cap raw RLIMIT_NOFILE in max_filedescriptors default (#2483) - #2486
Open
squidadm wants to merge 1 commit into
Open
Bug 5554: Cap raw RLIMIT_NOFILE in max_filedescriptors default (#2483)#2486squidadm wants to merge 1 commit into
squidadm wants to merge 1 commit into
Conversation
…-cache#2483) setMaxFD() updates Squid_MaxFD. Squid_MaxFD is then used to allocate various FD-indexed tables, including Comm's `fd_table`. When Config.max_filedescriptors is not set, Squid has to guess the maximum number of descriptors this instance can handle. Prior to these changes, that guess was based on OS-provided ulimits (RLIMIT_NOFILE). Using raw RLIMIT_NOFILE values to set Squid_MaxFD results in huge fd_table allocations that kill or incapacitate Squid when OS uses very large limits (e.g., Kubernets soft limit of 1073741816=2^30-8 effectively means "unlimited" and results in ~432 MB fd_table). We now cap such raw values at 100K which yields ~42 MB fd_table. Several other old problems were discovered and marked during this work. Those out-of-scope problems deserve dedicated fixes. N.B. setMaxFD() calls setrlimit(2) in some cases, but the primary/final setrlimit(2) call is in setSystemLimits(). That primary call sets the soft limit to Squid_MaxFD.
squid-anubis
added
the
M-failed-description
Collaborator
|
Cannot create a git commit message from PR title and description. Error while parsing future commit message title: Problematic parser input: Please see PR title and description formatting requirements for more details. This message was added by Anubis bot. Anubis will add a new message if the error text changes. Anubis will remove M-failed-description label when there are no corresponding failures to report. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
setMaxFD() updates Squid_MaxFD. Squid_MaxFD is then used to allocate
various FD-indexed tables, including Comm's
fd_table. WhenConfig.max_filedescriptors is not set, Squid has to guess the maximum
number of descriptors this instance can handle. Prior to these changes,
that guess was based on OS-provided ulimits (RLIMIT_NOFILE).
Using raw RLIMIT_NOFILE values to set Squid_MaxFD results in huge
fd_table allocations that kill or incapacitate Squid when OS uses very
large limits (e.g., Kubernets soft limit of 1073741816=2^30-8
effectively means "unlimited" and results in ~432 MB fd_table). We now
cap such raw values at 100K which yields ~42 MB fd_table.
Several other old problems were discovered and marked during this work.
Those out-of-scope problems deserve dedicated fixes.
N.B. setMaxFD() calls setrlimit(2) in some cases, but the primary/final
setrlimit(2) call is in setSystemLimits(). That primary call sets the
soft limit to Squid_MaxFD.