Pantagrule - Large Hashcat Rulesets Generated From Real-World Compromised Passwords

gargantuan hashcat rulesets generated from compromised passwords

Project maintenance warning: This project is deemed completed. No pull requests or changes will be made to this project in the future unless they are actual bugs or migrations to allow these rules to work with newer versions of hashcat.

Pantagrule is a series of rules for the hashcat password cracker generated from large amounts of real-world password compromise data. While Pantagrule rule files can be large, the rules are both tunable and perform better than many existing rule sets.

Pantagrule was generated using PACK's Levenshtein Reverse Path algorithm for automated rule generation (Kacherginsky, 2013). PACK's output was then sorted based upon the number of times PACK generated the rule to make the base ruleset. This process is similar to the rules generated by _NSAKEY for password cracking competitions in 2014 (_NSAKEY, 2014), however, Pantagrule was generated off a significantly larger set of passwords. Version 2 of Pantagrule was developed off of the publicly-available hashes.org "founds" corpus, a best-in-class public wordlist. This yields more transparent results than the original variant, which used a proprietary corpus containing 842,643,513 unique passwords.

When such large rulesets are fed through PACK, millions of rules result. However, since most of the rules generated appear only a handful of times, most of the useful rules are the ones that are most commonly generated by the algorithm. This repository contains a subset of rules generated by PACK whilst iterating through the existing corpus.

Optimised variants

In order to generate a second-pass optimisation of the rules against real-world data, the top one million generated rules was run against the Pwned Passwords NTLM list using the rockyou wordlist. Any rule that cracked a password was added to its own list and poorer-performing rules were discarded.

Four optimisation types were created:

popular.rule: pantagrule.1m run against the top 25,000,000 passwords of the HIBP set.
random.rule: pantagrule.1m run against 25,000,000 randomly selected passwords from the HIBP set.
hybrid.rule: A sorted list of a combination of the most successful popular and random rules, then cut in half, in an attempt to make a lighter, "balanced" ruleset that works across a larger sample set.
one.rule: A version of OneRuleToRuleThemAll in which the top performing hybrid rules are appended, and the list is truncated to the size of the dive rule set. Interestingly, there is only a couple-thousand-rule overlap with OneRuleToRuleThemAll and the Pantagrule rules, making the two strategies complementary. Pantagrule's one performs better than other known lists of this size, and it is recommended that you start with this ruleset before attempting one of the larger variants.

Pantagrule hashorg.v6

After the success of these large rulesets, an attempt was made of the inverse of the royce variant, in which the original Pantagrule methodology was used but both sets of data were different. Pantagrule now uses the public hashes.org "founds" list as its wordlist base for rule generation, and an optimisation pass was then made against the V6 NTLM list from Have I Been Pwned. Given the fully-public nature of the data used, it also allows the publication of raw reproducibility data, including pantagrule.v2.1m.rule, which are the top one million rules generated by this methodology. The data for V5 and V6 is the same for the top 25 million passwords.

For this version, the way one is generated has changed. To generate one, the full 1 million list was appended to OneRuleToRuleThemAll.rule and then the entire set calibrated on Pwned V6, vs. just appending rules and truncating.

Naming conventions for the rules have now changed to be of the format pantagrule.${corpus}.${trainingversion}.${extension}. This makes it easier to understand what the rule was optimised for. For example, for pantagrule.hashorg.v6.random, We used the random methodology with hashes.org as the basis for the rule generation, optimised on Pwned Passwords V6.

Original rules

Original rules were trained using the proprietary wordlist alongside the Pwned Passwords NTLM v5 set using rockyou.txt as a base. Since the "training data" and the validation data are the same, it would make sense to see them optimised for the V5 dataset.

The royce variants

Upon request of hashcat contributor Royce Williams, optimisations of the top one million rules were also run with the hashes.org founds list. This is due to the HIBP corpus being relatively dirty, and the hashes.org founds list being likely to yield a more practical ruleset for real-world cracking. These have been added as the royce variants. The royce optimisations appear to consist of marginally fewer rules overall, and random.royce is substantially more effective on a long tail of passwords than the original random. Performance did not increase over the existing rules on some variants, but given that the training and validation data of the original Pantagrule are both from the Pwned Passwords dataset, this does not seem surprising. Pantagrule royce variants exist in the rules/royce folder.

Performance vs. other commonly-used rules

In order to test any successes of the Pantagrule strategy against other rulesets, we will run validation data across the top 25 million passwords of Pwned Passwords V5 and the top 100 million passwords of Pwned Passwords V5 to get an understanding of rule effectiveness at cracking the "long tail" with each ruleset. The canonical rockyou.txt will be our dictionary and our baseline.

Original variant generation was done on an 8x 1070Ti rig running hashcat v5.1.0. The royce Pantagrule variants were created on a 4x Radeon VII rig running hashcat git build v5.1.0-1774-gf96594ef. The hashorg.v6 variants were created and validated (very slowly) on a single NVIDIA Tesla M4, a single 1070Ti, and hashcat v6.1.0.

In order to note rule performance against very common passwords, 0-25M is broken out into its own column. The RPP column is the rules per percent on the 100M dataset. This is calculated by using the formula rpp = Math.round(num_rules / (0_100m_percent - 6.450)). The higher this number, the more rules are run per percentage cracked. This helps realise the diminishing returns in rulesets and gives an idea of the amplified cost of running the rules on slower hashes.

Rules	Number of Rules	V5 25M	V5 100M	RPP
No Rules (just rockyou.txt)	0	16.549%	6.450%	N/A
pantagrule.private.v5.one	99,092	79.814%	69.417%	1,574
pantagrule.private.v5.hybrid	355,205	81.346%	73.372%	5,308
pantagrule.private.v5.popular	478,736	81.792%	73.544%	7,135
pantagrule.private.v5.random	616,236	81.687%	69.805%	8,828
pantagrule.hashorg.v6.one	99,092	74.500%	60.573%	1,831
pantagrule.hashorg.v6.hybrid	339,953	77.649%	68.341%	5,493
pantagrule.hashorg.v6.popular	514,416	80.668%	72.377%	6,931
pantagrule.hashorg.v6.random	638,773	80.603%	72.713%	8,614
pantagrule.private.hashorg.one.royce	99,092	79.618%	69.092%	1,582
pantagrule.private.hashorg.hybrid.royce	314,268	81.068%	73.082%	4,716
pantagrule.private.hashorg.popular.royce	420,984	81.386%	73.102%	6,316
pantagrule.private.hashorg.random.royce	592,235	81.659%	74.010%	8,766
best64	64	45.117%	24.985%	3
hob064	68	37.786%	19.773%	5
OneRuleToRuleThemAll	52,014	78.058%	64.541%	895
d3adhob0	57,548	51.274%	34.800%	2,030
dive	99,092	77.111%	63.314%	1,743
_NSAKEY V1	123,289	76.42%	64.121%	2,138
_NSAKEY V2	123,289	76.882%	64.472%	2,124

Conclusion

This work confirms the limitations of the PACK LRP algorithm originally witnessed by _NSAKEY on modern data sets when using the rockyou dictionary. While the LRP algorithm does generate rules that increase cracking percentage, it does so at a large increase in search space. For this reason, Pantagrule is most useful in cases where difficult cracking requires exotic rules.

It is important to note that if you can use PACK to generate rules based off of a specific corpus and then target your remaining hashes with it, you are likely to yield a greater cracking percentage than using one of these large rulesets. For example, Pantagrule V2 does not perform as well on PPv5 as the v5-calibrated ruleset.

Since the original Pantagrule release, these rules have proven themselves on multiple red team engagements at large technology companies and consultancies alike. The original pantagrule.1m list cracked 8% of the remaining HIBP hashes that had stood up to the corpus used to generate Pantagrule, the above common rule sets, a 7-character alphanumeric brute force, and KoreLogic's PathWell topologies.

As even the author of the One Rule to Rule Them All (Hunt, 2017) meta-rule states, there is no such thing as a rule that works better than others. Every use case is different, and every rule source may be one that helps you more than another on a specific hash dump or with a specific wordlist. Note that this data does not show what has been cracked; some rules have cracked hashes that other rules have not.

Citations

Rabelais, F. (1532). Pantagruel. Paris: Libr. générale française.
Kacherginsky, P. (2013). Automatic Password Rule Analysis and Generation. [online] Available at: https://medium.com/@iphelix/automatic-password-rule-analysis-and-generation-7d2574516e48 [Accessed 4 Oct. 2019].
_NSAKEY. (2014). NSAKEY/nsa-rules. [online] Available at: https://github.com/NSAKEY/nsa-rules [Accessed 4 Oct. 2019].
Hunt, W. (2017). One Rule to Rule Them All. [online] Available at: https://www.notsosecure.com/one-rule-to-rule-them-all/ [Accessed 4 Oct. 2019].