Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,5 +175,5 @@ Internal discovery runs a single masscan sweep (no source-port override) followe
- **Hostname support**: hostnames in the target file are resolved once at startup; nmap receives the original hostname (for SNI/vhost), masscan receives the resolved IP
- **IPv4-only, enforced at the edges**: the tool scans IPv4 exclusively (masscan/nmap invocations, target expansion, and address sorting all assume it). IPv6 is rejected rather than half-supported, in two places. (1) `_build_discovery_target_file()`'s `_parse_ranges()` skips any entry `ipaddress.ip_network()` resolves to a non-v4 network and prints the offending file, line number, and content — previously the v6 bounds were stored silently and only surfaced hundreds of lines later as `AddressValueError: ... (>= 2**32)` from `summarize_address_range()`, and only when an exclusions file happened to be configured. (2) The masscan/discovery XML parsers (`_parse_masscan_ping_xml()`, `_parse_nmap_sn_xml()`, `_run_masscan_batch()`) select `address[@addrtype='ipv4']` instead of the first `<address>` child, matching what the nmap-side parsers already did, so a dual-stacked host's IPv6 or MAC string can't enter `live_ips`/`port_ips` and become a masscan `-iL` target. Address sorting goes through `_ip_sort_key()`, which orders valid IPv4 numerically and sorts anything unparseable last instead of raising — the three former inline `tuple(int(o) for o in x.split('.'))` keys ran *after* a completed sweep, so one odd entry discarded the whole thing.
- **XML result parsing is per-element defensive**: every `etree.parse()` site guards the *walk* as well as the parse. Attributes are read with `.attrib.get(...)` and the element is skipped when the identifier is missing — never a bare `attrib['addr']` or `findall('address')[0]`, both of which raise `KeyError`/`IndexError` that `except etree.ParseError` does not catch. Those exceptions escaped the guard and discarded the results for *every other host* in the file (or, in `_host_elem_to_dict()`, lost `spoonmap_output.xml`/`.json` for the whole run) over one truncated element. `<script>` elements with no `id=` are filtered out of the comprehensions for the same reason. Where a fallback to the first `<address>` child is wanted after `address[@addrtype='ipv4']` misses (`generate_findings()`, `_scan_extra_sql_ports()`), it is a `None`-checked `find('address')`.
- **Firewall state table safety**: internal discovery caps masscan at `INTERNAL_DISCOVERY_MAX_RATE = 1000 pps`; at that rate with a 60 s half-open timeout, concurrent state entries peak at ~60 K regardless of target range size; for ranges above `INTERNAL_DISCOVERY_STATE_CEILING = 262_144` hosts the port list is trimmed from 10 to 5 to keep total packet volume bounded
- **Firewall state table safety**: internal discovery caps masscan at `INTERNAL_DISCOVERY_MAX_RATE = 1000 pps`; at that rate with a 60 s half-open timeout, concurrent state entries peak at ~60 K regardless of target range size; for ranges above `INTERNAL_DISCOVERY_STATE_CEILING = 262_144` hosts the port list is trimmed from 10 to 5 to keep total packet volume bounded. Separately and for the same reason, `mass_scan()` clamps a **Full** scan to `full_scan_rate` — 10000 pps External, 1000 pps Internal — since a single 1-65535 invocation fans out every port across every target at once. This cap applies *only* to `scan_type == 'Full'`; category and custom batched scans scan a handful of ports per invocation and always use the operator's full `max_rate`. The clamp prints a notice when it actually lowers the rate, because `main()`'s run summary echoes the *requested* `max_rate`: clamping silently made the summary contradict what masscan was told to do, and read as the operator's `--max-rate` having been ignored outright.
- **Honeypot/tarpit detection**: `mass_scan()` flags hosts open on ≥`HONEYPOT_OPEN_PORT_FRACTION` (90%) of scanned TCP ports (min sample `HONEYPOT_MIN_PORTS_SCANNED = 10`) as likely tarpits (LaBrea, portspoof) via `_flag_suspected_tarpits()`/`_report_suspected_tarpits()`, writing `discovery/suspected_tarpits.txt`. Separately, `_count_unmatched_service_ports()` reads `nmap_results/*.xml` and counts open ports whose `-sV` probe captured a `servicefp` (no signature match); `≥HONEYPOT_MIN_UNMATCHED_PORTS = 3` such ports on one host is consistent with decoys (e.g. Artillery) that return random data on full connect. Both signals surface as a single "Likely Honeypot / Decoy Host" MEDIUM finding in `generate_findings()`. `_flag_suspected_tarpits()` counts TCP ports only, skipping any `port_key` that starts with `U:`, so every loop that reconstructs a port key from a `live_hosts/portNN.txt` filename must run the stem through `_fname_port()` (`'U_53'` → `'U:53'`) — the raw stem was counted as TCP, skewing the open-port fraction, and printed as `Hosts Found on Port U_53`.
11 changes: 11 additions & 0 deletions spoonmap.py
Original file line number Diff line number Diff line change
Expand Up @@ -1668,6 +1668,17 @@ def mass_scan(scan_type, dest_ports, source_port, max_rate, target_file, exclusi
# most exposed to a live_hosts/ directory left over from a wider scope.
_report_out_of_scope_retained(full_results, scope_ranges, target_file)
return status_summary
# The run summary printed by main() shows the operator's requested
# max_rate, not full_scan_rate, so a silent clamp here reads as the rate
# having been ignored. Disclose it at the point of use.
if int(full_scan_rate) < int(max_rate):
print(_COLOR_ERROR
+ f'Notice: Full port scan rate capped at {full_scan_rate} pps '
f'(requested {max_rate}) — a 1-65535 sweep fans out every port '
f'across every target at once, and the {target_scan} cap keeps '
'firewall state tables bounded. Targeted port scans use the '
'full requested rate.'
+ _COLOR_RESET)
print(_COLOR_INFO + 'Full port scan: running masscan 1-65535 (no probe)...' + _COLOR_RESET)
full_results = _run_masscan_batch(['1-65535'], full_scan_rate, output_file,
target_file, source_port, exclusions_file,
Expand Down
23 changes: 23 additions & 0 deletions tests/test_spoonmap.py
Original file line number Diff line number Diff line change
Expand Up @@ -2059,6 +2059,29 @@ def test_full_scan_rate_capped_internal(self, tmp_path):
wait_secs=2,
)

def test_full_scan_warns_when_rate_is_capped(self, tmp_path, capsys):
"""A clamped rate is disclosed — the run summary shows the requested rate."""
spoonmap.output_path = str(tmp_path)
with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
mass_scan('Full', ['1-65535'], '88', '5000', '/fake/targets.txt', '')
out = capsys.readouterr().out
assert 'capped at 1000 pps' in out
assert 'requested 5000' in out

def test_full_scan_no_warning_when_rate_under_cap(self, tmp_path, capsys):
"""No notice when the operator's rate is already below the cap."""
spoonmap.output_path = str(tmp_path)
with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
mass_scan('Full', ['1-65535'], '88', '500', '/fake/targets.txt', '')
assert 'capped' not in capsys.readouterr().out

def test_category_scan_does_not_warn_about_full_scan_cap(self, tmp_path, capsys):
"""The cap applies only to Full — a batched scan uses the full rate silently."""
spoonmap.output_path = str(tmp_path)
with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
mass_scan('Category', ['22'], '88', '5000', '/fake/targets.txt', '')
assert 'capped' not in capsys.readouterr().out

def test_full_scan_writes_live_hosts_files(self, tmp_path):
spoonmap.output_path = str(tmp_path)
fake_results = {'22': {'10.0.0.5', '10.0.0.6'}}
Expand Down
Loading