John The Ripper - External Mode - Recover Partially Remembered Password

John the Ripper (JtR) is a well known security utility to crack passwords. In its usual use case JtR is used to brute force password hashes which requires access to the username and corresponding password hash. We cover the use of John the Ripper in our security courses such as Certified Ethical Hacker

Recover Partially Remembered Password

In this post I look at using JtR to recover a partially remembered password. In my case I needed to create some custom password generation rules and, as I had no access to the password hash, it was an encrypted file, I needed to use John to simply generate the passwords so I could use them in a script.

John The Ripper - Modes

John has 4 modes:

  • single - which attempts to guess passwords based on username and other GECOS information stored in /etc/passwd and /etc/shadow files. JtR expects its target password file to contain the username and hash together - i.e. a concatenation of the passwd and shadow file and a utility called "unshadow" to create a single file for you,
  • wordlist -  a dictionary of values is provided and JtR iterates over the word list hashing values and, optionally, applying transformation rules on the dictionary words to account for variations like uppercase or lowercase, character substitution and other common approaches to human based password generation,
  • Incremental - This is where JtR will generate passwords systematically in an attempt to brute force guess the password. This is extremely slow and, unless the password is short, will probably never finish in an acceptable amount of time,
  • external - This option allow the user to defined their own password generation strategies using a subset of c

Initially I was hoping the word list mode would meet my needs. I planned on putting the partially remember portion of the password in word list file and then adding a custom rule to generated the guesses. Reading and absorbing the documentation around John is a slow process, there are few references and mostly you need to read the configuration file to figure out how the rules work. After several hours I concluded that word list and rules would not work for me.

John The Ripper - EXTERNAL Mode

I did considered writing my own generator in Java or even JavaScript, since it wold just be a bunch of nested loops and fairly trivial to write but, hey, why not try external mode :)

External mode allows one to write rules for generating password guesses. There are three functions that can be implemented:

  • init() - set up your one time initialisation
  • filter() - function to determine if candidate password should be used. Hashing can be expensive so discarding candidate password you don't want to try speeds things up. This is useful for incremental mode
  • generate() - this function will generate a candidate password
  • restore() - restore the run once it has been interrupted. Basically need to save global state to enable restart

The generation function is probably the trickiest to write as you need to "return" a single candidate password each invoke and manipulate global variables to maintain state. If you wrote a script to do the guessing you would probably invoke the generate function one and have it loop over local variables and apply the guess on each iteration.

I find that much easier to reason about what I am dong than having JtR invoke the generate function each iteration. For the scant details on the syntax used for external mode see JtR EXTERNAL documentation here. Then have a look at the entries in /etc/john/john.conf.

Note: This is a subset of C which makes coding a little verbose. For a list of available global variable please see docs linked above.

External Mode - Rule for Partially Remembered Password

The requirement I had was to generate the last part of a partially remembered password. The user was fairly certain about the first part of the password but not the last part. She did have some notion of what the remaining letters might be.  Below is the EXTERNAL rule I ended up writing.

It take the first part of the password which the user is fairly certain about, then creates a list of candidate characters to append and then generates all possible combinations of the passwords in increasing length to a specified length. It may be useful as a basis for someone else in a similar predicament.

Note: I am sure there are some bugs here. If you find any let me know. Its hard to write unit tests for this.


int maxSuffixLen; //maximum number of "slots" to append
int options[5]; //candidate characters to append
int counter;//track number of passwords generated
int base;//number of options per "slot"
int currentSuffixLen;

void init(){
        //should be the same as the length of the options array
        //the max length of the suffix to calculate.

void generate(){

        //Check to see if we are at the end of the generation cycle        
        if (currentSuffixLen>=maxSuffixLen){

        //partially remembered password
        int rem;
        int index;
        //the position to insert the character at in the word.
        int pow;
        pow= currentSuffixLen-1;

        int denom;
        int i;

        int currentMax;
        int numplaces;
        numplaces= pow;

        while (numplaces>=0){
          int quot;
          if (quot!=0){
                rem = rem%denom;
          } else if (denom>1){
          } else if(rem<base){
        if (counter>=currentMax*base){
                currentSuffixLen=currentSuffixLen +1;

To use this is a script, i.e. not against a hashed password you can use a script similar to the following:

list=`john --external:JB --stdout`
for pwd in $list; do
	openssl  rsa -passin pass:$pwd -in  ./id_rsa -noout 2>/dev/null
	if [ $? -eq 0 ]; then
		echo "found $pwd"
		exit 0;
	echo $pwd
echo "not found!"
exit 1;



I don't think it's been called the "GECOS field" for, well, decades.

And the name of that OS changed to GCOS in 1971 when GE sold their computer business to Honeywell.