Feat/python crypto detection - #495
Conversation
n1ckl0sk0rtge
left a comment
There was a problem hiding this comment.
PR Review: Feat/python crypto detection (#495)
Linked Issues: none — PR has no Closes #N reference
Requirements Coverage: feature largely delivered, but 3 of the 8 advertised rule groups don't reach production (see MUST FIX 1, 4, 5, 7)
Architecture: CLEAN — no layer violations; rules import no com.ibm.mapper.*, all 8 holders follow the memoization contract
Domain Patterns: 8 blocking findings (crypto mis-mapping + dead detections)
Blast Radius: MEDIUM — the engine prefix-matching change is provably inert for Java/Go, but SignatureReorganizer sits on live Java and Go paths with a semantic change and zero Java/Go tests
Security: no secrets in fixtures, no new dependencies, no analysis-time IO — clean
Tests: 55 new tests, all green (mvn test -pl python -am → BUILD SUCCESS, 106 python tests, 0 failures); assertions genuinely exhaustive
Breaking Changes: EXTERNAL — MethodMatcher public constructor replaced; existing pyca curve detection regresses
MUST FIX (Blocking)
1. [Architecture] PythonCryptoPublicKey.rules() is never registered — all public-key detection is dead in production
python/src/main/java/com/ibm/plugin/rules/detection/PythonDetectionRules.java:81-87
Only 7 of the 8 pycrypto holders are in buildRules(). RSA.generate(), DSA.generate(), ECC.generate(), ECC.import_key(), ElGamal.generate() produce zero findings in a real scan. The 4 publickey tests pass only because they instantiate PythonCryptoPublicKey.rules() directly — they give false confidence.
Fix: add PythonCryptoPublicKey.rules().stream(), to the Stream.of(...).
2. [Domain] PycaCurveMapper drops SECP521R1 and SECP256K1 — regression for existing pyca users
mapper/src/main/java/com/ibm/mapper/mapper/pyca/PycaCurveMapper.java:70
PycaPrivateKeyContextTranslator previously handled both inline; it now delegates to this mapper, which has neither. Instead it has case "SECP512R1", "PRIME512V1", "P-512", "P512", "NIST P-512" -> new Secp521r1(...) — secp512r1 does not exist. So ec.SECP521R1() and ec.SECP256K1() in already-scanned Python code silently stop emitting their curve component.
Fix: rename the case set to "SECP521R1", "PRIME521V1", "P-521", "P521", "NIST P-521" and re-add case "SECP256K1" -> new Secp256k1(...).
3. [Domain] Ed25519/Ed448 conflated with X25519/X448 curves
mapper/src/main/java/com/ibm/mapper/mapper/pyca/PycaCurveMapper.java:72-75 — case "ED25519", "CURVE25519" -> Curve25519. EdDSA signature curves and Montgomery ECDH curves are distinct, and the repo already models them separately (Ed25519/Ed448 in PycaPrivateKeyContextTranslator:97-98). ECC.generate(curve='Ed25519') is now reported as an ECDH curve.
Fix: split the aliases. Note ECCTestFile.py:4 bakes the conflation into an assertion, so that marker needs updating too.
4. [Domain] bcrypt is detected but never translated — dead detection
python/src/main/java/com/ibm/plugin/rules/detection/pycrypto/kdf/PythonCryptoKDF.java:280,293 emit kind = "bcrypt", but PycaKeyDerivationContextTranslator has zero bcrypt cases and there is no Bcrypt model class. BcryptTestFile.py:6 admits it — # detected but not mapped — and BcryptTest.java:73 asserts nodes is empty. The test locks in the gap rather than covering the feature.
Fix: add a Bcrypt algorithm model + translator case, or drop BCRYPT/BCRYPT_CHECK until it exists.
5. [Domain] EdDSA import rules emit "ECC", which no translator handles — dead detection
python/src/main/java/com/ibm/plugin/rules/detection/pycrypto/publickey/PythonCryptoPublicKey.java:217,230 build contexts with algorithm = "ECC", but PycaPublicKeyContextTranslator handles only DH|RSA|DSA|ELGAMAL and PycaPrivateKeyContextTranslator handles EC, not ECC. Both eddsa.import_private_key / import_public_key rules never translate.
Fix: use "ED25519"/"ED448", or add an ECC case.
6. [Domain] "shuflle" typo
python/src/main/java/com/ibm/plugin/rules/detection/pycrypto/random/PythonCryptoRandom.java:58 — Crypto.Random.random.shuffle() is never detected. No test exists for PythonCryptoRandom at all, which is why it wasn't caught.
7. [Domain] The four common PBKDF2 forms are commented out of buildRules()
python/src/main/java/com/ibm/plugin/rules/detection/pycrypto/kdf/PythonCryptoKDF.java:322-325 — only the 5-arg PBKDF2_WITH_COUNT_AND_HASH is active, so PBKDF2(password, salt, 32) and PBKDF2(password, salt, 32, count=100000) are undetected. The private constants remain in the file (lines 149-222).
Fix: enable or delete. Same pattern at PythonCryptoCipher.java:311 (references PKCS1_OAEP_NEW_WITH_HASH, a constant that does not exist) and PythonCryptoPublicKey.java:76-88,118-130,194-206.
8. [Breaking Change] MethodMatcher public constructor replaced, not overloaded
engine/src/main/java/com/ibm/engine/detection/MethodMatcher.java:72
MethodMatcher(String[], String[], List<String>) was mutated in place into a 4-arg prefixMatch form. docs/LANGUAGE_SUPPORT.md:459 points extension authors at this class — this is a source- and binary-incompatible removal for out-of-tree consumers. Nothing in-repo breaks.
Fix: keep the 3-arg array constructor delegating this(..., false).
SHOULD FIX (Non-blocking)
9. [Domain] All 14 KDF rules use .inBundle(() -> "PyCa") instead of "PyCrypto" — PythonCryptoKDF.java (14/14; every other pycrypto class correctly uses "PyCrypto"). PyCryptodome KDF findings will be attributed to the pyca library in the CBOM.
10. [Blast Radius] SignatureReorganizer mutates a live, aliasable children map — mapper/src/main/java/com/ibm/mapper/reorganizer/rules/SignatureReorganizer.java:200-201,263-264. The ConcurrentModificationException fix (values().forEach(n::put); getChildren().clear();) iterates and clears the live internal map — and Algorithm's copy constructor aliases it (Algorithm.java:39: this.children = algorithm.getChildren()), so clear() can wipe a different node's children. Both methods are on live Java (JavaReorganizerRules:58) and Go (GoReorganizerRules:64,66,68) paths, and this PR adds zero Java/Go tests.
Fix: iterate a defensive copy; add a multi-child regression test for Java and Go.
11. [Scope] mapper/ciphersuites.json + JsonCipherSuites.java are unrelated build artifacts — mapper/download-cipher-suites.sh curls the file from ciphersuite.info and creatCipherSuiteClass.py regenerates the Java. The semantic diff is clean (348 → 351 entries: 3 ASCON suites added, 0 removed, 0 modified), but this has nothing to do with Python crypto detection.
Fix: git checkout main -- mapper/ciphersuites.json mapper/src/main/java/com/ibm/mapper/mapper/ssl/json/JsonCipherSuites.java; land the ASCON suites separately.
12. [Domain] DES_3 is fully subsumed by DES_2 → duplicate findings — PythonCryptoCipher.java:99-112 vs :84-97. Since prefixMatchesParameters accepts actual >= expected, DES.new(key, DES.MODE_CBC, iv) matches both rules. Fix: delete DES_3.
13. [Domain] ChaCha20 / ChaCha20-Poly1305 get no Encrypt/Decrypt nodes — PythonCryptoCipher.java:186-208 end with .withoutDependingDetectionRules() while every sibling attaches List.of(ENCRYPT, DECRYPT).
14. [Domain] pkcs1_15 signatures lose their PKCS#1 v1.5 padding — PythonCryptoSignature.java:81 → ValueActionFactory("RSA"), indistinguishable from a raw RSA signature. Compare pss → "RSA-PSS" → RSAssaPSS. The padding branch at PycaSignatureContextTranslator.java:82 is still case "PKCS1V15" -> Optional.empty(); // TODO.
15. [Domain] DSS rules lack withOtherParameters() — PythonCryptoSignature.java:123-152 require exactly 2 params, so the documented DSS.new(key, 'fips-186-3', encoding='der') is missed. The ECDSA twins do use it. (DSS-vs-ECDSA discrimination itself is correct — it keys off the first parameter's type, so there is no double counting.)
16. [Domain] HMAC/CMAC bind the algorithm to the 2nd positional parameter, which is msg — PythonCryptoMac.java:42-64. The real signatures are HMAC.new(key, msg=b'', digestmod=None) and CMAC.new(key, msg=b'', ciphermod=None). HMAC.new(key, data, SHA256) is missed.
17. [Domain] Poly1305 / KMAC128 / KMAC256 registered as digests, not MACs — PythonCryptoHash.java:41-68. This contradicts PythonCryptoMac.java:66-78, where the Poly1305 MAC rule was deliberately disabled.
18. [Domain] All hash rules use .withAnyParameters(), dropping size/variant — PythonCryptoHash.java:83-92. SHA512.new(truncate='256') reports plain SHA-512 even though PycaDigestMapper already has SHA512_224/SHA512_256 ready; BLAKE2b/s lose their digest size.
19. [Domain] The HPKE rule likely cannot fire — PythonCryptoCipher.java:269-285 uses two positional parameters, but PyCryptodome's HPKE.new() is keyword-only. HPKETestFile.py uses invented syntax (from Crypto.Protocol.HPKE.AEAD import AES128_GCM — AEAD is a class, not a submodule), so the green test does not prove real-world matching. Worth confirming against a real install.
20. [Tests] Three new components have no test at all — PythonCryptoRandom (both rules), PycaRandomContextTranslator (0% coverage; only reachable from it), and PycaCurveMapper (20 branches, no test anywhere in mapper/src/test/). The missing curve test is exactly what would have caught MUST FIX 2.
21. [Tests] The Cryptodome.* branch is structurally untested — zero of the 55 new .py files use the Cryptodome prefix. DSS_CRYPTODOME and ECDSA_CRYPTODOME exist solely for it and are never exercised. Given the PR's headline claim of pycryptodomex support, this is the largest coverage gap.
22. [Tests] The new core-engine matching mode has no engine test — withOtherParameters() / SOME_WITH_REMAINDER / prefixMatchesParameters appear nowhere under any src/test/. Used 15 times, verified only indirectly through Python integration tests.
23. [Architecture] Unchecked (PublicKeyEncryption) casts on Optional<? extends Algorithm> — PycaKeyContextTranslator.java:79, PycaPublicKeyContextTranslator.java:79, PycaPrivateKeyContextTranslator.java:74. Safe only by accident today; a ClassCastException here aborts the Sonar analysis of the file. Fix: narrow PycaCurveMapper.parse to Optional<EllipticCurveAlgorithm>.
24. [Architecture] The key-algorithm switch is duplicated across three translators and is already drifting — ELGAMAL was added to two of them, FERNET/EC_IMPORT to only one. Fix: extract an IMapper into mapper/.../mapper/pyca/.
25. [Docs] withOtherParameters() is absent from the documented builder grammar — docs/DETECTION_RULE_STRUCTURE.md:172,178. Third parties following this doc will not discover the new DSL step.
26. [Breaking Change] MethodMatcherSerializer does not emit prefixMatch — exported rule JSON cannot distinguish prefix from exact arity, misrepresenting all 15 new rules.
27. [Domain] PycaSecretContextTranslator is now dead code — unwired from PythonTranslator (replaced by PycaKeyContextTranslator for KeyContext) but still present. Confirm no pyca regression, then delete.
28. [Domain] bcrypt cost is log2 of rounds, mapped straight to ITERATIONS — PythonCryptoKDF.java:269-282 reports cost=10 as 10 iterations instead of 1024.
29. [Docs] Broken README link — README.md:39: [PyCryptodome(x)][https://www.pycryptodome.org/] is reference-link syntax with no defined reference, so GitHub renders the literal text. Use (...) instead. Also revert the trailing-whitespace-only change on line 38.
SUGGESTIONS
scrypt'sris a block-size factor (block = 128·r bytes), reported as raw bytes —PythonCryptoKDF.java:258-261- Append
SOME_WITH_REMAINDERat the end of the enum rather than inserting it — safe today (noordinal()/serialization use anywhere), defensive convention only - Guard
prefixMatchesParametersagainst an emptyexpectedTypeslist (it would match every invocation) — currently unreachable, but one refactor away PycaCipherContextTranslator.java:109-122compares rawasString()while neighbouring switches use.toUpperCase().trim()Crypto.Protocol.DH.key_agreementis hard-coded to"ECDH"even though the X25519/X448 import rules feed itPycaCurveMapperhas phantom aliases (PRIME192V1/PRIME224V1/PRIME384V1are not real X9.62 names) and partial brainpool coverage — suggests the list was extrapolated rather than taken from PyCryptodome's curve table- Delete the ~20 lines of commented-out original code in
SignatureReorganizer.java:194-205,257-268
What's good here
Worth calling out explicitly — several things were done carefully:
- The 95-file rename is flawless. Zero stale FQN references repo-wide, including
rules/, plugin resources, and rule metadata; all 96PythonCheckVerifier.verify(...)path strings resolve. The pyca move is a pure rename with 0 changed lines in the rule bodies. - The memoization contract is followed perfectly across all 8 new classes, including all four suppliers in
PythonCryptoPublicKey. - Layering is respected — no rule class imports
com.ibm.mapper.*;PycaCurveMapperis correctly language-agnostic. - Translator dispatch ordering is correct and safe —
IDetectionContext.is()uses exact class equality rather thanisAssignableFrom, and the newKeyContextbranch was placed after the subtype branches anyway. - Test assertions are genuinely exhaustive (12–46 assertions per class, covering both the detection store and the translated nodes) — no stubs.
BcryptTest's empty-node assertion is honest documentation of a gap, not a cheat. - The broad
ENCRYPT/DECRYPT/SIGN/VERIFYrules are correctly excluded from top-level registration, so there is no repo-wide match on every.encrypt()call. prefixMatchis provably inert for Java/Go — gated behind the new opt-in builder step and used only inpycrypto/.
Bottom line: the reorganization and test discipline are solid, but the 8 blocking issues mean the feature does not fully work as advertised — one entire rule class is unregistered, three rule groups are dead detections, and the shared curve mapper regresses existing pyca output. The engine and mapper/ciphersuites.json changes would be better split into separate PRs.
This review was generated by AI (Claude). Findings may contain errors — please verify before acting on them.
… DSL - Add SOME_WITH_REMAINDER to CapturedParameterScope so a rule matches calls whose actual arity is >= the declared arity - Expose withOtherParameters() on every terminal builder stage in IDetectionRule; implement in DetectionRuleBuilderImpl - MethodMatcher: add boolean prefixMatch field and prefixMatchesParameters(); keep 3-arg array constructor as a backward-compatible delegate (prefixMatch=false) - MethodMatcherSerializer: emit prefixMatch boolean field in exported JSON - DetectionRuleStore: append prefixMatch flag to matcher ID to prevent silent deduplication of rules that differ only in arity mode Signed-off-by: san-zrl <san@zurich.ibm.com>
Signed-off-by: san-zrl <san@zurich.ibm.com>
…natureReorganizer Algorithm(Algorithm,Class) and Key(Key,Class) assigned the source node's live children map by reference; mutating the re-kinded copy corrupted the original node's children. Use new HashMap<>(src.getChildren()) instead. SignatureReorganizer.moveNodesFromUnder*: snapshot node.getChildren() into an ArrayList before iterating and remove each child individually via removeChildOfType() to avoid ConcurrentModificationException on the live map. Signed-off-by: san-zrl <san@zurich.ibm.com>
…yca mappers PycaCurveMapper: centralises curve-string to model mapping. Correctly splits Edwards curves (ED25519->Edwards25519, ED448->Edwards448) from Montgomery curves (CURVE25519->Curve25519, CURVE448->Curve448). Fixes SECP521R1 aliases (was SECP512R1) and re-adds the missing SECP256K1 case. PycaKeyBasedAlgorithmMapper: extracts the RSA/DSA/DH/ElGamal/Fernet algorithm switch that was duplicated across the three key-context translators. PycaCipherMapper: adds DES, RC2, Salsa20, AES128_GCM/AES256_GCM, RC4 alias, CHACHA20_POLY1305 alias. PycaDigestMapper/PycaMacMapper: adds MD2, MD4, RIPEMD-160, KMAC, TupleHash, cSHAKE, Keccak, KangarooTwelve for PyCryptodome rules. RIPEMD: adds (asKind, RIPEMD) copy constructor for MAC usage. Signed-off-by: san-zrl <san@zurich.ibm.com>
Relocate all existing pyca rule classes (aead, asymmetric, fernet, hash, kdf, keyagreement, mac, padding, symmetric, wrapping) into the new detection/pyca/ sub-package to match the pycrypto/ layout and avoid future naming collisions. No logic changes. Signed-off-by: san-zrl <san@zurich.ibm.com>
New detection-rule classes under detection/pycrypto/:
cipher - AES, DES, 3DES, Blowfish, CAST5, RC2, RC4, ChaCha20,
ChaCha20-Poly1305, Salsa20, PKCS1-OAEP, PKCS1-v1.5, HPKE
hash - MD2/4/5, SHA-1/2/3, RIPEMD-160, SHAKE, BLAKE2b/s, KMAC,
TupleHash, cSHAKE, Keccak, KangarooTwelve, Poly1305
mac - HMAC (2-arg and 3-arg), CMAC (2-arg and 3-arg)
kdf - PBKDF1 (5 arity variants), PBKDF2 (5 arity variants),
scrypt, HKDF, SP800-108
keyagreement - DH.key_agreement, X25519/X448 import helpers
publickey - RSA, DSA, ECC, ElGamal generate/import (RSA/DSA/ECC reached
as dependent rules from Signature; only ElGamal is top-level)
random - Crypto.Random.get_random_bytes, Crypto.Random.random methods
signature - PKCS1v15 (emits RSA-PKCS1V15), PSS, DSS, ECDSA, EdDSA
All rules use .inBundle(() -> "PyCrypto") for correct CBOM attribution.
Signed-off-by: san-zrl <san@zurich.ibm.com>
…lugin PythonDetectionRules: register all 8 new PyCryptodome rule bundles (Hash, Mac, Random, Cipher, PublicKey, Signature, KDF, KeyAgreement). PythonTranslator: replace dead PycaSecretContextTranslator with new PycaKeyContextTranslator; wire PycaRandomContextTranslator for PRNGContext; move KeyContext handling after Private/PublicKey so more-specific contexts are checked first. PythonReorganizerRules: add four moveNodesFromUnderFunctionalityUnderParent rules for Sign/Verify under Signature and PSS. PycaKeyContextTranslator (new): handles KeyContext for RSA/DSA/EC/ElGamal/ Fernet keys; delegates to PycaKeyBasedAlgorithmMapper and PycaCurveMapper. PycaRandomContextTranslator (new): maps PRNG to PseudorandomNumberGenerator. PycaPrivateKeyContextTranslator: delegate curve mapping to PycaCurveMapper; add ElGamal and EC_IMPORT cases; use PycaKeyBasedAlgorithmMapper. PycaPublicKeyContextTranslator: add ElGamal; add Curve branch via PycaCurveMapper; use PycaKeyBasedAlgorithmMapper. PycaCipherContextTranslator: add MODE_* aliases; PKCS1_OAEP/PKCS1_v1_5/ HPKE ValueAction cases. PycaSignatureContextTranslator: add RSA-PKCS1V15, DSS, ECDSA, EDDSA cases. PycaKeyDerivationContextTranslator: add PBKDF1/PBKDF2/HKDF root nodes; IterationCount/SaltSize translation; pycrypto-pbkdf* digest pa IterationCount/SaltSize translation; pycrypto-pbkdf* digest pa Iterationn to Algorithm. Signed-off-by: san-zrl <san@zurich.ibm.com>
Mirror the production-code reorganisation. No assertion changes; only package declarations and fixture paths updated. Signed-off-by: san-zrl <san@zurich.ibm.com>
Covers: cipher (AES, DES, 3DES, Blowfish, CAST5, RC2, RC4, ChaCha20, ChaCha20-Poly1305, Salsa20, PKCS1-OAEP, PKCS1-v1.5), hash (MD2/4/5, SHA-1/2/3, RIPEMD-160, BLAKE2b/s, TupleHash128, cSHAKE256), KDF (PBKDF1, PBKDF2, scrypt, HKDF), MAC (HMAC, CMAC), key agreement (DH, X25519/X448), public key (RSA, DSA, ECC, ElGamal), signature (PKCS1v15, PSS, DSS, ECDSA, EdDSA). Signed-off-by: san-zrl <san@zurich.ibm.com>
Signed-off-by: san-zrl <san@zurich.ibm.com>
…yca mappers PycaCurveMapper: centralises curve-string to model mapping. Correctly splits Edwards curves (ED25519->Edwards25519, ED448->Edwards448) from Montgomery curves (CURVE25519->Curve25519, CURVE448->Curve448). Fixes SECP521R1 aliases (was SECP512R1) and re-adds the missing SECP256K1 case. PycaKeyBasedAlgorithmMapper: extracts the RSA/DSA/DH/ElGamal/Fernet algorithm switch that was duplicated across the three key-context translators. PycaCipherMapper: adds DES, RC2, Salsa20, AES128_GCM/AES256_GCM, RC4 alias, CHACHA20_POLY1305 alias. PycaDigestMapper/PycaMacMapper: adds MD2, MD4, RIPEMD-160, KMAC, TupleHash, cSHAKE, Keccak, KangarooTwelve for PyCryptodome rules. RIPEMD: adds (asKind, RIPEMD) copy constructor for MAC usage. KeyAgreementReorganizer: add REPLACE_ECDH_WITH_X25519_WHEN_CURVE25519 and REPLACE_ECDH_WITH_X448_WHEN_CURVE448 rules to convert a generic ECDH node into the correct XDH algorithm when both key children carry the matching curve. Signed-off-by: san-zrl <san@zurich.ibm.com>
Relocate all existing pyca rule classes (aead, asymmetric, fernet, hash, kdf, keyagreement, mac, padding, symmetric, wrapping) into the new detection/pyca/ sub-package to match the pycrypto/ layout and avoid future naming collisions. No logic changes. Signed-off-by: san-zrl <san@zurich.ibm.com>
Delete the original flat detection/{aead,asymmetric,fernet,hash,kdf,
keyagreement,mac,padding,symmetric,wrapping} trees and the superseded
PycaSecretContextTranslator now that everything has been moved to
detection/pyca/ and replaced by the new translators.
Signed-off-by: san-zrl <san@zurich.ibm.com>
253117b to
87c5284
Compare
|
Hi @n1ckl0sk0rtge, Thanks for the review. I modified the code in response and pushed my changed . Here's what I did: Must FixM1: PythonCryptoPublicKey.rules() was redesigned — it now only contains ElGamal. The are marked as obsolete keys and should be detected stand-alone. A class-level comment documents this. RSA_GENERATE, DSA_GENERATE, ECC_GENERATE, etc. are intentionally not top-level rules. They are reached as dependent rules from PythonCryptoSignature — for example, PKCS1V15 and PSS call PythonCryptoPublicKey.RSARules(), DSS call PythonCryptoPublicKey.DSARules(), and ECDSA, EDDSA and DH_KEY_AGREEMENT call PythonCryptoPublicKey.ECCRules(). M2: The typo — SECP512R1 → SECP521R1 (and all its aliases) — fixed
M3: Fixed.
M4: Fixed. Removed bcrypt rule and test for the moment. M5: Fixed. Both detection rules now emit "EC" instead of "ECC" so that detections produce translated output. Tests M6: Fixed the typo. M7: Fxed. All five PBKDF2 variants are now active in buildRules(): PBKDF2, PBKDF2_WITH_HASH, M8: Fixed as prescribed. SHOULD FIXS9: All KDF rules use now .inBundle(() -> "PyCrypto") as suggested. S10: Fixed, but no multi-child regression tests for Java and Go.
S11: No change S12: DES_3 rule removed. Update of Cipher rules AES, DES, DES3, BLOWFISH, CAST, ARC2. S13: Fixed. Both rules now end with .withDependingDetectionRules(List.of(ENCRYPT, DECRYPT)), consistent with all other cipher rules. S14: Fixed.
S15: Fixed. Both DSS (line 140) and DSS_CRYPTODOME (line 156) now have .withOtherParameters() after the typed key parameter, with a comment explicitly listing what the extra args cover. This means DSS.new(key, 'fips-186-3'), DSS.new(key, 'fips-186-3', encoding='der'), and any other arity variant all match. The fix is symmetric with ECDSA (line 172) and ECDSA_CRYPTODOME (line 188), which already had it. S16: Fixed. The original single-rule approach — which bound the algorithm to the 2nd positional parameter and thus confused msg with the hash/cipher module — was replaced with a two-rule pattern for both HMAC and CMAC. S17: "KMAC128", "KMAC256", and "Poly1305" were wrongly detected as hashes and emitted as if they were digests. S18: Not addressed, accepted as a limitation.
S19: Removed attempted detection of HPKE and corresponding test. The case is similar to M4 (Bcrypt). S20, S21, S22: Missing tests not addressed. S23: Narrowed PycaCurveMapper.parse to Optional, cast exception now impossible. Removed cast in PycaKeyContextTranslator. The other two casts are necessary. S24: Fixed. Created PycaKeyBasedAlgorithmMapper and wired it into PycaKeyContextTranslator, PycaPrivateKeyContextTranslator and PycaPublicKeyContextTranslator replacing the drifting switch statements. S25: Fixed. Added withOtherParameters() to the grammar and explained it in the text below. S26: Fixed.
S27: Fixed. Removed dead PycaSecretContextTranslator..java S28: Void. Bcrypt removed in response to M4. S29. Fixed broken link and trailing whitespace in README.md. SuggestionsS30: Left as is |
Summary
This PR adds detection support for the PyCrypto / PyCryptodome library (
Crypto.*) and reorganises the existing Python detection rule infrastructure so that bothpycaandpycryptorules share the same translator pipeline.The rules also work for
pycryptodomexwhich exhibits the exact same API aspycryptodome. The only difference is the import path (Cryptographyinstead ofCrypto). Detection rules check for both possible import paths in forObjectTypes. Rule duplication was avoided by relaxing the type checking (withMethodParameter uses ANY, parameters detected during translation). Only one duplicate rule.Here's a result that this PR produces with cbomkit when scanning mastercard python client API (no findings before).

What changed
1 · New PyCrypto / PyCryptodome detection rules
Eight new detection-rule classes cover the full PyCryptodome API surface:
PythonCryptoCipherPythonCryptoHashPythonCryptoKDFPythonCryptoKeyAgreementPythonCryptoMacPythonCryptoPublicKeyECCRules()for ECDSA dependency), ElGamalPythonCryptoSignaturePythonCryptoRandom2 · Shared PyCA translators for PyCrypto
pycryptodetections are wired into the existing pyca context translators rather than duplicating translation logic. Three new translators were added to fill gaps:PycaKeyContextTranslator— unified key material handling for both librariesPycaPublicKeyContextTranslator— public-key extraction shared by pyca and pycrypto RSA/DSA/ECCPycaRandomContextTranslator— random context (new, used byPythonCryptoRandom)3 · Code structure reorganisation
All pyca detection rules and their test input files moved into a dedicated
pyca/sub-package, mirroring the newpycrypto/layout:Test classes
.javaand test input.pyfiles follow the same structure undersrc/test/files/rules/detection/pyca/andpycrypto/.PythonDetectionRules.javaandPythonReorganizerRules.javaupdated to include all new rule registrations.4 · Mapper additions
PycaCurveMapperEllipticCurvemodelsPycaDigestMapperPycaMacMapperPycaCipherMapperSignatureReorganizer5· Performance
Added memoization to hot translation paths to avoid redundant re-computation during multi-rule traversals.
Tests
55 new JUnit tests under
python/src/test/java/…/pycrypto/— one per detection scenario — covering approximately 80 % of the new detection rules:41 existing pyca tests updated to reflect the restructured package paths.
Directory structure change
detection/aead/Pyca*.javadetection/pyca/aead/Pyca*.javadetection/asymmetric/Pyca*.javadetection/pyca/asymmetric/Pyca*.javadetection/hash|kdf|mac|padding…detection/pyca/hash|kdf|mac|padding…test/files/rules/detection/*.pytest/files/rules/detection/pyca/*.pydetection/pycrypto/**/*.javatest/files/rules/detection/pycrypto/**/*.pyTranslator additions & changes
PycaKeyContextTranslatorPycaPublicKeyContextTranslatorPycaRandomContextTranslatorPythonCryptoRandomPycaCipherContextTranslatorPycaKeyDerivationContextTranslatorPycaSignatureContextTranslatorPycaPrivateKeyContextTranslatorPycaKeyContextTranslatorPycaDigestContextTranslatorPycaKeyAgreementContextTranslatorPycaMacContextTranslator