-
Notifications
You must be signed in to change notification settings - Fork 3
Add 'osism reset facts' command to clear the Ansible fact cache #2389
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
berendt
wants to merge
4
commits into
main
Choose a base branch
from
implement/issue-2388-reset-facts
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
f97e158
Add 'osism reset facts' command to clear the fact cache
berendt 8d0c51e
Add unit tests for 'osism reset facts'
berendt ac80bc4
Assert single delete attempt in limited reset error test
berendt c21be3f
Reject empty --limit in 'osism reset facts'
berendt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import json | ||
| import subprocess | ||
|
|
||
| from cliff.command import Command | ||
| from loguru import logger | ||
| from redis.exceptions import RedisError | ||
|
|
||
| from osism import utils | ||
| from osism.utils.inventory import get_hosts_from_inventory, get_inventory_path | ||
|
|
||
|
|
||
| class Facts(Command): | ||
| """Reset (clear) the cached Ansible facts. | ||
|
|
||
| By default the whole fact cache is flushed. Use ``--limit`` to clear | ||
| only the facts of selected hosts or groups. The command does not | ||
| gather new facts; the cache is rebuilt on the next Ansible run that | ||
| collects facts. | ||
| """ | ||
|
|
||
| def get_parser(self, prog_name): | ||
| parser = super(Facts, self).get_parser(prog_name) | ||
| parser.add_argument( | ||
| "-l", | ||
| "--limit", | ||
| type=str, | ||
| help="Limit the reset to selected hosts or groups (Ansible host pattern)", | ||
| ) | ||
| return parser | ||
|
|
||
| def take_action(self, parsed_args): | ||
| if parsed_args.limit is not None and not parsed_args.limit.strip(): | ||
| logger.error("--limit must not be empty.") | ||
| return 1 | ||
| if parsed_args.limit: | ||
| return self._reset_limited(parsed_args.limit) | ||
| return self._reset_all() | ||
|
|
||
| def _reset_all(self): | ||
| removed = 0 | ||
| try: | ||
| cursor = 0 | ||
| while True: | ||
| cursor, batch = utils.redis.scan( | ||
| cursor, match="ansible_facts*", count=100 | ||
| ) | ||
| if batch: | ||
| utils.redis.delete(*batch) | ||
| removed += len(batch) | ||
| if cursor == 0: | ||
| break | ||
| except RedisError as exc: | ||
| logger.error(f"Failed to reset Ansible fact cache: {exc}") | ||
| return 1 | ||
|
|
||
| logger.info(f"Removed cached facts for {removed} host(s)") | ||
| return 0 | ||
|
|
||
| def _reset_limited(self, limit): | ||
| try: | ||
| result = subprocess.run( | ||
| [ | ||
| "ansible-inventory", | ||
| "-i", | ||
| get_inventory_path("/ansible/inventory/hosts.yml"), | ||
| "--list", | ||
| "--limit", | ||
| limit, | ||
| ], | ||
| capture_output=True, | ||
| text=True, | ||
| timeout=30, | ||
| ) | ||
|
|
||
| if result.returncode != 0: | ||
| logger.error( | ||
| f"Error loading inventory (rc={result.returncode}): " | ||
| f"{result.stderr}" | ||
| ) | ||
| return 1 | ||
|
sourcery-ai[bot] marked this conversation as resolved.
|
||
| except subprocess.TimeoutExpired: | ||
| logger.error("Timeout loading inventory.") | ||
| return 1 | ||
|
|
||
| try: | ||
| inventory = json.loads(result.stdout) | ||
| except json.JSONDecodeError as exc: | ||
| logger.error(f"Failed to parse inventory output: {exc}") | ||
| return 1 | ||
|
|
||
| hosts = get_hosts_from_inventory(inventory) | ||
|
|
||
| if not hosts: | ||
| logger.warning("No hosts matched the given limit.") | ||
| return 0 | ||
|
|
||
| keys = [f"ansible_facts{host}" for host in hosts] | ||
| try: | ||
| deleted = utils.redis.delete(*keys) | ||
| except RedisError as exc: | ||
| logger.error(f"Failed to reset Ansible fact cache: {exc}") | ||
| return 1 | ||
|
|
||
| logger.info(f"Removed cached facts for {deleted} host(s)") | ||
| return 0 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,212 @@ | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| """Tests for the ``osism reset facts`` command. | ||
|
|
||
| These cover the two reset paths and their edge cases: flushing the whole | ||
| ``ansible_facts*`` cache (including the empty-cache no-op), restricting the | ||
| reset to the hosts a ``--limit`` pattern resolves to, and the error contracts | ||
| for a failed inventory load and an unreachable Redis. | ||
| """ | ||
|
|
||
| import subprocess | ||
| from unittest.mock import MagicMock, patch | ||
|
|
||
| import pytest | ||
| from redis.exceptions import RedisError | ||
|
|
||
| from osism.commands import reset | ||
|
|
||
|
|
||
| def _make(): | ||
| return reset.Facts(MagicMock(), MagicMock()) | ||
|
|
||
|
|
||
| def _parse(*args): | ||
| return _make().get_parser("test").parse_args(list(args)) | ||
|
|
||
|
|
||
| @pytest.fixture | ||
| def mock_redis(): | ||
| """Provide a mock Redis client wherever the command resolves it. | ||
|
|
||
| ``osism.utils.redis`` is a lazily-initialised module attribute that opens | ||
| a real connection on first access, so patch both the attribute and its | ||
| initialiser to keep the test offline. | ||
| """ | ||
| client = MagicMock() | ||
| with patch("osism.utils._init_redis", return_value=client), patch( | ||
| "osism.commands.reset.utils.redis", client, create=True | ||
| ): | ||
| yield client | ||
|
|
||
|
|
||
| # --- flush-all path --- | ||
|
|
||
|
|
||
| def test_facts_flushes_all_keys_when_no_limit(mock_redis, loguru_logs): | ||
| mock_redis.scan.return_value = ( | ||
| 0, | ||
| [b"ansible_factsnode1", b"ansible_factsnode2"], | ||
| ) | ||
|
|
||
| rc = _make().take_action(_parse()) | ||
|
|
||
| assert rc == 0 | ||
| mock_redis.scan.assert_called_once_with(0, match="ansible_facts*", count=100) | ||
| mock_redis.delete.assert_called_once_with( | ||
| b"ansible_factsnode1", b"ansible_factsnode2" | ||
| ) | ||
| assert any("2 host(s)" in r["message"] for r in loguru_logs) | ||
|
|
||
|
|
||
| def test_facts_succeeds_and_skips_delete_when_cache_empty(mock_redis, loguru_logs): | ||
| mock_redis.scan.return_value = (0, []) | ||
|
|
||
| rc = _make().take_action(_parse()) | ||
|
|
||
| assert rc == 0 | ||
| mock_redis.delete.assert_not_called() | ||
| infos = [r for r in loguru_logs if r["level"] == "INFO"] | ||
| assert any("0 host(s)" in r["message"] for r in infos) | ||
|
|
||
|
|
||
| def test_facts_returns_nonzero_on_redis_error(mock_redis, loguru_logs): | ||
| mock_redis.scan.side_effect = RedisError("connection refused") | ||
|
|
||
| rc = _make().take_action(_parse()) | ||
|
|
||
| assert rc == 1 | ||
| mock_redis.delete.assert_not_called() | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("Failed to reset Ansible fact cache" in r["message"] for r in errors) | ||
|
|
||
|
|
||
| # --- limited path --- | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("limit", ["", " "]) | ||
| def test_facts_empty_limit_returns_nonzero_and_touches_nothing( | ||
| mock_redis, loguru_logs, limit | ||
| ): | ||
| rc = _make().take_action(_parse("-l", limit)) | ||
|
|
||
| assert rc == 1 | ||
| mock_redis.scan.assert_not_called() | ||
| mock_redis.delete.assert_not_called() | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("--limit must not be empty." in r["message"] for r in errors) | ||
|
|
||
|
|
||
| def test_facts_limit_deletes_only_selected_hosts(mock_redis): | ||
|
sourcery-ai[bot] marked this conversation as resolved.
|
||
| mock_redis.delete.return_value = 1 | ||
| ok = MagicMock() | ||
| ok.returncode = 0 | ||
| ok.stdout = "{}" | ||
|
|
||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch("osism.commands.reset.subprocess.run", return_value=ok), patch( | ||
| "osism.commands.reset.get_hosts_from_inventory", | ||
| return_value=["node1", "node2"], | ||
| ): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 0 | ||
| mock_redis.scan.assert_not_called() | ||
| mock_redis.delete.assert_called_once_with( | ||
| "ansible_factsnode1", "ansible_factsnode2" | ||
| ) | ||
|
|
||
|
|
||
| def test_facts_limit_returns_nonzero_when_inventory_fails(mock_redis, loguru_logs): | ||
| failed = MagicMock() | ||
| failed.returncode = 1 | ||
| failed.stderr = "boom" | ||
|
|
||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch("osism.commands.reset.subprocess.run", return_value=failed): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 1 | ||
| mock_redis.delete.assert_not_called() | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("Error loading inventory" in r["message"] for r in errors) | ||
| assert any("boom" in r["message"] for r in errors) | ||
|
|
||
|
|
||
| def test_facts_limit_returns_nonzero_on_invalid_inventory_json(mock_redis, loguru_logs): | ||
| ok = MagicMock() | ||
| ok.returncode = 0 | ||
| ok.stdout = "{not valid json" | ||
|
|
||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch("osism.commands.reset.subprocess.run", return_value=ok): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 1 | ||
| mock_redis.delete.assert_not_called() | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("Failed to parse inventory output" in r["message"] for r in errors) | ||
|
|
||
|
|
||
| def test_facts_limit_returns_nonzero_when_inventory_times_out(mock_redis, loguru_logs): | ||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch( | ||
| "osism.commands.reset.subprocess.run", | ||
| side_effect=subprocess.TimeoutExpired("ansible-inventory", 30), | ||
| ): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 1 | ||
| mock_redis.delete.assert_not_called() | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("Timeout loading inventory." in r["message"] for r in errors) | ||
|
|
||
|
|
||
| def test_facts_limit_warns_and_succeeds_when_no_hosts_match(mock_redis, loguru_logs): | ||
| ok = MagicMock() | ||
| ok.returncode = 0 | ||
| ok.stdout = "{}" | ||
|
|
||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch("osism.commands.reset.subprocess.run", return_value=ok), patch( | ||
| "osism.commands.reset.get_hosts_from_inventory", return_value=[] | ||
| ): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 0 | ||
| mock_redis.delete.assert_not_called() | ||
| warnings = [r for r in loguru_logs if r["level"] == "WARNING"] | ||
| assert any("No hosts matched the given limit." in r["message"] for r in warnings) | ||
|
|
||
|
|
||
| def test_facts_limit_returns_nonzero_on_redis_error(mock_redis, loguru_logs): | ||
| mock_redis.delete.side_effect = RedisError("connection refused") | ||
| ok = MagicMock() | ||
| ok.returncode = 0 | ||
| ok.stdout = "{}" | ||
|
|
||
| with patch( | ||
| "osism.commands.reset.get_inventory_path", | ||
| return_value="/ansible/inventory/hosts.yml", | ||
| ), patch("osism.commands.reset.subprocess.run", return_value=ok), patch( | ||
| "osism.commands.reset.get_hosts_from_inventory", | ||
| return_value=["node1", "node2"], | ||
| ): | ||
| rc = _make().take_action(_parse("-l", "control")) | ||
|
|
||
| assert rc == 1 | ||
| # The limited path deletes all matched keys in a single batched call, so a | ||
| # RedisError must surface after exactly one delete attempt without retrying. | ||
| assert mock_redis.delete.call_count == 1 | ||
| errors = [r for r in loguru_logs if r["level"] == "ERROR"] | ||
| assert any("Failed to reset Ansible fact cache" in r["message"] for r in errors) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if parsed_args.limit:treats-l ""as "no limit" and falls through to_reset_all(), flushing the entire fact cache.The realistic trigger is a script calling
osism reset facts -l "$PATTERN"where$PATTERNexpands empty — the operator means to scope the reset but wipes every host instead.Reject an empty/whitespace-only limit explicitly before branching, e.g.:
Don't fix this by passing
""through toansible-inventoryinstead: I verified against the pinnedansible-core==2.19.3thatansible-inventory --limit ""returns all hosts (rc=0).Please add a test asserting
-l ""returns non-zero and never callsredis.delete/scan.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reject an empty/whitespace-only --limit with a non-zero exit before branching in take_action, and added a parametrized test asserting -l "" and -l " " return 1 and never call redis.scan/delete.
Files changed:
osism/commands/reset.py,tests/unit/commands/test_reset.pyPosted by planwerk-agent 394730b address with Claude