free web page hit counter


Obtaining the Files: Compiled Jtr and Tools

Prerequisites: *Note an example ntds.dit/SYSTEM file is in the above download

  • Administrator Access
  • Shadow Copy administration tools like those built in to Windows (VSSAdmin)
  • If using 2008 or greater use NTDSUtil (This utility needs restrictive ACL's or be deleted if it is not used)
  • John the Ripper(compiled) or use a full Cygwin environment| Linux host.
  • LibEseDb, this is the tool that initially extracts the AD database.
  • NTDSXtract(compiled or full Cygwin+Python environment)

Grabbing the NTDS.dit, as well as the SYSTEM file is easy with VSSAdmin, and there is no need to use the "hacking" tools like PWDump, FGDump or Cain&Abel. VSSAdmin is built-in and easy to use, but you can also use VBScript(WMI) and PowerShell as well.

Server 2008 (or greater)

NTDSUtil.exedoes everything you need in a one line command:

ntdsutil.exe "activate instance ntds" "ifm" "Create Full C:\Temp\123" quit quit

As seen in the screenshot below, an entire copy of the NTDS.dit file, as well as the SYSTEM and SECURITY files (Registry Hives) are coped for you (or for the attacker as the case may be).

Screenshot of NTDSUtil in action:

Server 2003 (or greater)

Use VSSAdmin to create shadow copies of the files you need. *Your paths may vary

Vssadmin create shadow /For=c:
Copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\Windows\NTDS\ntds.dit c:\temp
Copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\ C:\Windows\System32\config\SYSTEM c:\temp
Vssadmin delete shadows /shadow={Some-Random-Looking-Character-ID}

Remember to insert the ID that was created with you ran the "create shadow" command, and use that ID to delete that specific Shadow copy instance.

Screenshot of VSSAdmin in action:

Alternative methods for grabbing the NTDS.dit, SYSTEM and or SAM are:

Offline copies, either from a VM snapshot and mounted as a secondary drive in another VM

Read off the disk directly using CD/DVD/USB boot disk

Physically removing and reading the drive as a secondary drive

  • Using tools like PWDump7 . PWDump7 uses an alternative NTFS driver, and ignores/bypasses the kernel's lock on those files.
  • 3rd party Shadow Copy tools like Hobocopy (does not trigger AV)

Memory and injection tools, like Pwdump6/FGdump and Cain&Abel also work, but most of those mentioned above are typically flagged by AV vendors as hacking tools, and could lead to false positives later on if you have to forensically inspect your PC/Domain Controller later on. Was that tool used by you or the bad guy, or both? Not to mention that you may have to disable AV during the process; this leaves you exposed during that time. Caches for various versions of these tools can be found on the Openwall (JtR) site: http://passwords.openwall.net/microsoft-windows-nt-2000-xp-2003-vista-7 There was also a PWDumpX that worked well at the time (2008), I can't say how well it works now.

Handling the Files:

Rename the NTDS.dit and SYSTEM files to account for the domain name(s) e.g. ops.NTDS.dit, ops.SYSTEM. Once you have a copy of the NTDS.dit and SYSTEM (or SAM) files, make sure you transfer them as securely as you can. Over the network you should encrypt them in a compressed archive (Winzip, 7zip etc...) first and then copy them to the extraction and cracking destination. You should also make sure you skip the recycle-bin, hold shift+delete to ensure they do not simply go to the recycle bin when deleting them from the copies from the DC or the PC. Again delete the files on the Domain Controller using shift+delete after you have copied them off.

Extracting the Hashes: NTDS.dit

Place the NTDS.dit and SYSTEM files in the folder you plan to use for the extraction process. You can compile the libesedb sources yourself (./configure &&make) or use the ones I provided with this document. To extract the NTDS.dit into a plain-text data structure:

esedbexport.exe <ntds.dit>

Screenshot of EseDbExport.exe:

Next using DsUsers.exe (or DsUsers.py) is needed to extract the encrypted hashes with the Syskey Boot-Key found in the SYSTEM file/hive. DsUsers then sorts and collates the each users AD attributes and settings as well based on the LinkTable. These paths are not absolute, you can find "datatable5" or some other variations like that, check your files names/paths :

dsusers.exe ntds.dit.export\datatable.3 ntds.dit.export\link_table.5 --passwordhistory SYSTEM >domain.txt

Screenshot of DsUsers.exe:

Using Grep and Sed can help you clean up the output to only hashes if you like. You can actually use John the Ripper directly on the DsUsers output file if you wish:

grep :::$ ops.output.txt | sed 's/^\s*//' |sort|uniq >ops-hashes.txt

Using JtR:

As of May 8th 2014 the Bleeding-Jumbo branch of JtR comes with an Autoconf script, this script makes current machine decisions based on what is installed and what hardware you are compiling the binary on. If your CPU does not support Intel's AVX or AES-NI, then the binary won't either unless you override that setting at compile time. For our purposes, this simple command will build JtR, or you can use the standalone version that should accompany this document.

Use Git clone to grab the latest version, or go to GitHub and grab a Zip of the latest, extract from the archive, then CD to the SRC directory and run configure + make like this:

./configure --enable-mpi &&  make clean && make

It is beyond the scope of this paper to show you how to compile/install JtR using Cygwin or other environments. You can find pre-built one's accompanying this document and here: 64-bit M$, various, and bleeding-edge.

John modes overview:

Single Crack:

This is the mode you should start cracking with. It will use the login names, "GECOS" / "Full Name" fields, and users' home directory names as candidate passwords, also with a large set of mangling rules applied. Since the information is only used against passwords for the accounts it was taken from (and against password hashes which happened to be assigned the same salt), "single crack" mode is much faster than wordlist mode. This permit's for the use of a much larger set of word mangling rules with "single crack", and their use is always enabled with this mode. Successfully guessed passwords are also tried against all loaded password hashes just in case more users have the same password.

Wordlist :

This is the simplest cracking mode supported by John. All you need to do is specify a wordlist (a text file containing one word per line) and some password files. You can enable word mangling rules (which are used to modify or "mangle" words producing other likely passwords). If enabled, all of the rules will be applied to every line in the wordlist file producing multiple candidate passwords from each source word.

The wordlist should not contain duplicate lines. John does not sort entries in the wordlist since that would consume a lot of resources and would prevent you from making John try the candidate passwords in the order that you define (with more likely candidate passwords listed first). However, if you don't list your candidate passwords in a reasonable order, it'd be better if you sort the wordlist alphabetically: with some hash types, John runs a bit faster if each candidate password it tries only differs from the previous one by a few characters. Most wordlists that you may find on the Net are already sorted anyway.

Some useful Wordlist mode examples:

john.exe NT-Password.txt -format=nt -w=dictionary.txt -rules=jumbo -session=wrdlist1
john.exe NT-Password.txt -format=nt -w=dictionary.txt -rules=nt -session=wrdlist2
john.exe NT-Password.txt -format=nt -w=dictionary.txt -rules=single -session=wrdlist3

Incremental :

This is the most powerful cracking mode; it can try all possible character combinations as passwords. However, it is assumed that cracking with this mode will never terminate because of the number of combinations being too large (actually, it will terminate if you set a low password length limit or make it use a small charset), and you'll have to interrupt it earlier.

That's one reason why this mode deals with trigraph frequencies, separately for each character position and for each password length, to crack as many passwords as possible within a limited time.

To use the mode you need a specific definition for the mode's parameters, including password length limits and the charset to use. These parameters are defined in the configuration file sections called [Incremental:MODE], where MODE is any name that you assign to the mode (it's the name that you will need to specify on John's command line). You can either use a pre-defined incremental mode definition or define a custom one.

As of version 1.8.0, pre-defined incremental modes are "ASCII" (all 95 printable ASCII characters), "LM_ASCII" (for use on LM hashes), "Alnum" (all 62 alphanumeric characters), "Alpha" (all 52 letters), "LowerNum" (lowercase letters plus digits, for 36 total), "UpperNum" (uppercase letters plus digits, for 36 total), "LowerSpace" (lowercase letters plus space, for 27 total), "Lower" (lowercase letters), "Upper" (uppercase letters), and "Digits" (digits only). The supplied .chr files include data for lengths up to 13 for all of these modes except for "LM_ASCII" (where password portions input to the LM hash halves are assumed to be truncated at length 7) and "Digits" (where the supplied .chr file and pre-defined incremental mode work for lengths up to 20). Some of the many .chr files needed by these pre-defined incremental modes might not be bundled with every version of John the Ripper, being available as a separate download.

Some useful Incremental mode examples:

john.exe NT-Password.txt -format=nt -incremental=digits -session=digits
john.exe NT-Password.txt -format=nt -incremental=ascii -session=Ascii
john.exe NT-Password.txt -format=nt -incremental=alnum -session=Alnum

The first command will try to guess all digit passwords, saving it's progress to the digits.rec file so you can pick up where an interrupted session stopped. The others are the same, Ascii consists of all 96 printable ascii characters. Alnum is lowercase and digits only.

External:

You can define an external cracking mode for use with John. This is done with the configuration (john.conf) file sections called [List.External:MODE], where MODE is any name that you assign to the mode. The section should contain program code of some functions thatJohn will use to generate the candidate passwords it tries. The functions are coded in a subset of C and are compiled by John at startup when you request the particular external mode on John's command line.

Some very useful External mode examples:

john.exe NT-Password.txt -format=nt -external=keyboard -session=keyb

The above will try keyboard passwords, and save its progress in a session file called keyb.rec. That way you can use the restore command to pick up where it left off if need be (john.exe -restore=keyb)

John.conf, john.local.conf and other settings:

The default settings that come with John are quite good, but they can easily be added and subtracted to as well. The default location for password hashes is the John.pot file (get it... hash, pot, eh eh...). You can specify alternate pot files, using -pot= filename. Then a .pot will be created with that name and the cracked hashes are stored there. Some of the specific settings I like to change I've included in the john.local.conf file, these will override the defaults found in john.conf. You can specify other external files to be read, or change the default dictionary from password.lst to be something you've created.

You should always create a session name (-session=some_name), that way the default one doesn't get used or overwritten. Session names have to be unique while John is working, you can reuse names, but if you have a session named "qwerty" going and you try to start another one on the same machine it will tell you that session is in use. Almost all modes can be resumed using the -restore= switch.

Parallelism and John:

Most password hashing in the past was fast, very fast, and NTLM is no exception. Some however are slow, very very ver y slow, and there are some optimizations that John has that will severely increase the speed of your cracking. NTLM is not aided by the OMP parallel capabilities like slower hashes are. But that is not to say that there are not large workloads that can't be split-up or distributed!

Fork and Node are very powerful tools with John; they do not however work very well at this time on windows. They do allow you to distribute a large workload out to multiple processors and even other computers! Simply using -fork=4 will split whatever command you just typed, out into four different threads. You want to make sure your CPU has at least that many cores or threads available. So if you wanted to use -wordlist=password.lst -rules=nt -fork=8, that would split the "CaMeLCaSiNg" work out to eight threads, and should make that task go eight times as fast it would if you just used one thread. Essentially John is going to divide that task up 8 ways, and each will being work.

Node is another way you can further distribute the work. This option is intended to allow for some trivial manually-configured parallel and distributed processing. For example, to split the workload across two nodes (which could be machines, CPU cores, etc.), you'd specify "--node=1/2" on one invocation of John and "--node=2/2" on the other. Using fork and node together is easy, but you should take into account how many nodes there are when using fork. Each thread becomes a node... For example, to use two similar 8-core machines you may run "--fork=8 --node=1-8/16" on one of the machines and "--fork=8 --node=9-16/16" on the other.

There is plenty of documentation on the mailing lists, the wiki and in the documents in the DOC directory that John comes with.

http://openwall.info/wiki/john
http://www.openwall.com/lists/john-users/
http://www.openwall.com/lists/john-dev/

Tips and Advice:

Some hashes are the "fast" type and some the "slow" type. For Windows AD and the SAM they are fast. The "slowest" hashes that John cracks are the Office-2007 and higher ones, and the EFS files. Kerberos, LM, NTLM and RC4 (older office) are all very fast, and none have parallel support in John beyond Fork/Node.

Look into how the rules in John.conf work, and read the syntax inside the README's included with John. Look into using OMP as well as MPI when using slow hashes. The Markov mode I do not fully understand, but it's basically a rating system for password difficulty. John can get a boost from GPU's as well for many (not all) hashes, I do not have any experience in that area. I do know that OclHashcat is faster than John on every format John's GPU support has. GPU's can get you better results than a CPU without question, I simply have no GPU that currently out performs my CPU :)

Policies Affect on Key space:

Password policies don't typically subtract a significant amount from the possible keyspace, but they can actually hurt your security if they are too aggressive. There are 6634204312890625 possible passwords if the length is 8, and using all possible types of characters: Upper, Lower, Digit and Special (space is included). If you require one of each type (U,L,D,S) you've reduced the possible password key space of length 8 passwords by 55%.

The math is: 6634204312890625 vs. 3025989069143040. You can see these effects on this John Wiki page: http://openwall.info/wiki/john/policy. Using only 3 classes isn't much better at that length, and it can also vary depending which policy you use:

5405597778540960 = Upper, Lower, Specials [19% reduction in key space for length 8]
3380063031465120 = Digit, Lower (same if you sub Upper here), Specials [50% reduction in key space for length 8]
3185644980510720 = Digit, Upper, Lower [52% reduction in key space for length 8]

Longer passwords do not significantly reduce the key space, and arguably the policies matter less. Longer passwords are harder to find in the first place using wordlist and brute force methods. The length contributes more to the strength of the password than the policies do in my opinion.

Wordlists:

There are so many places and ways to get wordlists, some of my favorites are SkullSecurity,Openwall, IMDB,CDDB, Twitter,InsidePro, Purdue andso muchmore! There are also lots of othertools and scripts out there that can be used. Also accompanying this document is the WordMachine from 16systems, which has abdicated their many useful tools.

Other useful items:

John comes with many something2john scripts and binaries, these attempt to grab the hashes from various sources or files like, winzip, 7zip, Vnc(pcap), M$ Office, FireFox master password etc... Typically you call them using the: something2john file_in_question >output.txt

Loopback:

The loopback feature is one of the best recent features in John. It will read the default john.pot, or an alternate one, and only use the plain-text passwords contained there as a wordlist:

john.exe NT-Password.txt -format=nt -loop -rules=jumbo -session=loopback

The above will replay all the cracked passwords in john.pot, and apply the Jumbo rule set. You can apply any rule set you wish; jumbo is a connection of 4 rule sets actually. The above does not iterate over itself, whatever passwords were in the pot at the time you ran the command are the only ones it will try. If you keep getting cracks using it, keep running it against itself over and over, it works very well, especially when people only change 1 small thing in their password histories.

Custom.chr files:

This one is like loopback, but you can also make your own pot file that you will then use to make an incremental .chr file with. John's current default .chr files are based off of the Rockyou.txt password leak. You prefix each plaintext password with a colon, and write them to a file. Then you use the -make-charset= switch to create a .chr file based on your current john.pot or one that you have in mind.

sed 's/^/:/' < rockyou.txt  >rockyou.pot (use sed to artificially create a pot file)
john.exe -pot=rockyou.pot -make-charset=rockyou.chr

You can apply a filter, present in john.conf (or john.local.conf) that restricts the candidates to something like Alpha only, Digits only or perhaps those that match a Password Policy! There are external modes that already do Policy based filtering, I am not sure a .chr file is best for that.

john.exe -pot=rockyou.pot -make-charset=rockyou.chr -ext=filter_lanman

The above will use the external filter called Filter_Lanman in john.conf, only using uppercase characters, digits and specials.

Pipe and Stdin:

The -pipe and -stdin switches are wonderful when you want to use other tools to feed John candidate passwords. The only differences I know of is that -pipe allows the use of the -rules switch, while -stdin can only work with what it is given.

john.exe --stdout=11 -incremental=alpha -session=stdout | john.exe NT-Password.txt -format=nt -pipe -rules=cap_num -session=piped-in

Tools that come with John:

  • Unique.exe is typically faster than sort|uniq, and very easy to use:
  • cat file.txt | unique -cut=12 output.txt (unique typically expects stdin input)
    unique -inp=file.txt -cut=12 output.txt (same thing just not using cat)
    
  • Unshadow.exe, this will take the passwd and /etc/shadow files and output the password hashes from a *nix system.
  • TgtSnarf.exe, this will grab Kerberos ticket from a pcap file and give you the hash output you need to use.
  • Mkvcalcproba.exe, this will give you password ratings that you can use with the Markov modes. I do not know how to use this :) See the wiki.
  • SIPDump.exe, this will again parse a pcap file and you will probably associate SIP traffic with IP Phones (VOIP) or Proxy server traffic, and give you the output needed to sic John upon.

What else can John do?

What can't it do? How many hash types does John support? Innumerable, the dynamic code (see DYNAMIC in DOC) supports hundreds if not thousands by itself... At the time I wrote this document, there are 184 types you can specify using the -format= switch, some overlap, some support more than one version of a hash type! Determining the correct hash type to use can be difficult if you are not using a something2john script/binary, but John does attempt to tell you what it thinks the type is.

You can use John in a variety of ways, not to mention using it's -stdout switch to itself!

john.exe -incremental=alnum -stdout -session=stdout | john.exe -pipe -rules=nt -session=pipe_nt

The above will output John's incremental alphanumeric output into John's pipe and apply a camelcase rule set to it. You can combine a john.local.conf incremental mode to only try passwords from length 8 to 9 so that you don't waste your time on passwords that are shorter:

    [Incremental:Alnum_8-9]
    File = $JOHN/alnum.chr
    MinLen = 8
    MaxLen = 9
    CharCount = 62

Then use the previous command above, only change the -incremental= to alnum_8-9:

john.exe -incremental=alnum_8-9 -stdout -session=stdout | john.exe -pipe -rules=nt -session=pipe_nt

The WIKI, included documentation and Mailing lists ( users|dev) are your best sources for information:

https://www.google.com/search?q=site%3Aopenwall.com+search_term_here

https://www.google.com/search?q=site%3Aopenwall.info%2Fwiki%2Fjohn+search_term_here

More Examples:

john.exe LM-Password.txt -format=lm -external=dumbforce_lm -session=dumbforce_lm
john.exe NT-Password.txt -format=nt -external=dumbforce_lcase -session=dumbforce_lcase

The above examples use the external mode to bruteforce uppercase only, and lowercase only, using the traditional aa, ab, ac, ad, ae... "dumb force" methods. The LM version using uppercase only because all LM passwords are Upper case, you can also use the dubmforce_allcase to use every possible case if your hash supports that.

john.exe NT-Password.txt -format=nt -external=repeats_lowercase -session=rep_lc
john.exe NT-Password.txt -format=nt -external=sequence -session=seq
john.exe NT-Password.txt -format=nt -external=subset -session=subset
john.exe NT-Password.txt -format=nt -external=atleast1-generic -session=at1-generic

Read the John.conf file looking for the external mode names, and see what each does. You can use -stdout just to have a look as well; you do not need a password file in that case:

john.exe -stdout -external=sequence
Press 'q' or Ctrl-C to abort, almost any other key for status
    !
    !"
    "#
    #$
    $%
    %&
    &'
    Etc...
    6789:;<=>?@ABCDEF
    789:;<=>?@ABCDEFG
    89:;<=>?@ABCDEFGH
    9:;<=>?@ABCDEFGHI
    :;<=>?@ABCDEFGHIJ
    ;<=>?@ABCDEFGHIJK
    <=>?@ABCDEFGHIJKL
    =>?@ABCDEFGHIJKLM
    >?@ABCDEFGHIJKLMN
    ?@ABCDEFGHIJKLMNO
    @ABCDEFGHIJKLMNOP
    ABCDEFGHIJKLMNOPQ
    Etc...
    3456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghij
    456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijk
    56789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijkl
    6789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklm
    789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmn
    89:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmno
    9:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnop
    :;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopq