Skip to content

Add separately configurable liveness and readiness probes - #166

Open
feldgendler wants to merge 1 commit into
livekit:masterfrom
feldgendler:probes-independently-configurable
Open

Add separately configurable liveness and readiness probes#166
feldgendler wants to merge 1 commit into
livekit:masterfrom
feldgendler:probes-independently-configurable

Conversation

@feldgendler

Copy link
Copy Markdown

Closes #116, and supersedes #161 — same configurability, plus a change to the liveness default.

The chart hardcodes the same GET / check for both livenessProbe and readinessProbe, so the two can't be tuned apart without forking the chart. Both are values now; setting either to ~ omits that probe.

Liveness also moves from GET / to a TCP check on the same port. GET / returns 406 once the node's stats are more than 4s stale, and on a redis deployment those stats are refreshed by a keepalive that round-trips through redis, so a redis outage longer than about 35s fails liveness on every node at the same moment and restarts the whole fleet — which does nothing to bring redis back (livekit/livekit#4663). Readiness keeps GET /, which is where staleness belongs: the node stops taking new sessions and starts again on its own once redis recovers. Every other probe field keeps the value Kubernetes was already defaulting it to, except livenessProbe.initialDelaySeconds, which goes from 0 to 10 to cover a cold start.

Also adds the chart's first unit tests (helm-unittest) and a CI step to run them before ct lint, since the liveness default is easy to regress by accident. tests/ is in .helmignore so it isn't packaged.

One question: I left Chart.yaml alone, since version bumps here happen in release commits — but nothing reaches operators until the version moves, so say the word if you'd rather this carried 1.11.1.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Readiness probe must not be the same as the liveness one

2 participants