Showing posts with label Hashing. Show all posts
Showing posts with label Hashing. Show all posts

Monday, April 10, 2017

Performance of MD5, SHA1, SHA256, SHA384, and SHA512 with C#

Recently I ran a few performance calculations to see the differences of performance of MD5, SHA1, SHA256, SHA384, and SHA512. I ran the tests on my box with one code set. I compiled my code as a 32-bit program and as a 64-bit program. Besides the performance of each hashing algorithm, I wanted see the difference in 32-bit and 64-bit compiles would make.


My testing method consisted of the following method.
  • CPU: Intel(R) Core (TM) i7-4700MQ CPU 2.40GHz
  • OS: Windows 7, 64-bit Operating System
  • Memory: 32.0 GB (31.6 GB usable)
  • Hash string size was 1K.
  • Ran in release mode, (not inside of Visual Studio, language used was C#)
  • Each hash string method was called 10,000 times with a different string to be hashed.
  • Time was accumulated in CPU ticks.
  • I ran each the program three times.
 

The one thing that stood out to me was SHA-256. I was surprised by the performance, the small difference between 32 and 64-bit programs and the actual time it took to create an SHA-256 hash. After spending some time on the internet I came up with the following.
  




 
SHA-256 algorithm generates an almost-unique, fixed-size 256-bit (32-byte) hash. So even on a 64-bit machine where the word size is larger to push more data thru the transformations algorithm because SHA-256 is using a fix 32-byte data you do not see the larger difference between a 32 and 64 bit processors.  One interesting note is Microsoft has not updated its core base class from which all implementations of cryptographic hash algorithms. The HashAlgorithm class is still is using 32-bit transformblocks. I am curious why Microsoft hasn't change to use a 64-bit transformblock. Would that improve the performance level? I would have thought Microssoft would have a 64-bit version of SHA256.




The largest takeaway here is "Do not use MD5 or SHA1". The difference in performance isn't going to hurt your application as much as using an outdated hashing algorithm will.



Code.

  Calling method to create a Hash string. All hashing followed the same template.


         static Random random = new Random();
         static Int64 MaxIterations = 10000;
         static Int16 TotalHashLength = 1024;
              static void MD5()
              {
                  MD5Hash md5Hash = new MD5Hash();
                  for (int i = 0; i < MaxIterations - 1; i++)
                  {
                      md5Hash.CalculateHash(RandomString(TotalHashLength));
                  }
                  file.WriteLine("MD5 , {0}", md5Hash.TotalElapseTime().ToString());
              }
 

 Each time an hash was created a different random string was created using lower, upper case, number and special printable characters.

 
              static string RandomString(int length)
              {
                  const string chars = "Aa0BbCcDd1+EeFf2_GgHh3(IiJj4)KkLl5*MmNn6&OoPp7^QqRr8%SsTt9#UuVv@WwXxYyZz!";
                  StringBuilder sb = new StringBuilder();
                  for (int i = 0; i < length; i++)
                  {
                      sb.Append(new string(Enumerable.Repeat(chars, 1).Select(s => s[random.Next(s.Length)]).ToArray()));
                  }
                  return sb.ToString();
             }
   

 Main hashing calculation for MD5, other hashing methods follow this template.


              private Stopwatch sw;
              private MD5 md5;
              private TimeSpan ElapseTime;
    
               public void CalculateHash(string input)
               {
                  sw = Stopwatch.StartNew();
                  md5 = MD5.Create();
                  byte[] inputBytes = System.Text.Encoding.ASCII.GetBytes(input);
                  byte[] hash = md5.ComputeHash(inputBytes););
                  ElapseTime += sw.Elapsed;
                  sw.Stop();
               }




For the complete code solution see my github page https://github.com/lwconklin/Hashing

Monday, June 17, 2013

Eric Snowden and OWASP Hashing & Salt


Sorry for being dark so long. I am being pulled in several directions at once. My main priorities right now is…

  • Work. They pay the bills.
  • OWASP Code Review Guide. I am the co-leader and project support of this project. It is one of OWASP Flag Ship products. 
  • Tulsa .Net Users Group. This year we are doing a coding contest every quarter sponsor by Inceed (http://www.inceed.com/index_sm.html) see (http://codeshootout.com). I am the contest master who comes up with the contest objectives, rules, etc. with some help from friends.


On OWASP Code Review Guide 2.0 we are re-vamping the book published in 2008 to refresh it, expand on it and build on the great platform created by the first book.

OWASP is a great organization that is always looking for good people to volunteer some effort on many great projects, like the Code Review Guide. Anyone intested??? www.owasp.org

The Code Review Guide are wiki articles.  Authors post these articles to OWASP main wiki. Once we have the content needed we will combine these articles into a book, with review process being done by OWASP and professional editor. I am also going to post some of the articles here in my blog. This article is one I wrote on Hashing and Salting. Interesting enough Bruce Schneier in his monthly crypto-gram newsletter has an article on password cracking which I though fitted nicely with my article on hashing and salting.  You can read his newsletter on the web at…
<http://www.schneier.com/crypto-gram-1306.html> I have written on this subject before in my blog but I feel this is the type on information that can and should be repeated. See also (http://arstechnica.com/security/2013/05/how-crackers-make-minced-meat-out-of-your-passwords/ )


Eric Snowden
One point I would like to make before we get into the Hashing stuff is Mr.Schneier comments and essay on whistleblowers like Eric Snowden. My question to Mr. Schneier is "How does the whistleblower know if they are exposing a true abuse in power or hurting our national security?". I am in favor of whistleblowers exposing abuse in power by our government or any government official but I am also not in favor of hurting our national security. I also don't want to give up every freedom I have to be "safe" but I realize the governments need to keep secrets. The discussion I would like to see out of this mess is a clear-cut understandable checks and balance on our government in the context private/personal information gathering. How they can be held liable and what are the limits on how intrusive they can be into our private lives and communications. How do we know they are staying in those limit and who are the gatekeepers? Sorry Eric, Manning and Julian Assange but I want better then the three of you.



Code review Guide – Hashing and Salting
Introduction
A cryptographic hash algorithm; also called a hash "function" is a computer algorithm designed to provide a random mapping from an arbitrary block of data (string of binary data) and return a fixed-size bit string known as a “message digest” and achieve certain security.
Cryptographic hashing functions are used to create digital signatures, message authentication codes (MACs), other forms of authentication and many other security applications in the information infrastructure. They are also used to store user passwords in databases instead of storing the password in clear text and help prevent data leakage in session management for web applications. The actual algorithm used to create a cryptology function varies per implementation (SHA-256, SHA-512, etc.)


The code reviewer needs to be aware of three main things when reviewing code that uses cryptographic hashing functions.

* Legality of the cryptographic hashing functions if the source code is being exported to another country.

* The life cycle of the cryptographic hashing function being used.

* Basic programming of cryptographic hashing functions.

Legal
In the United States in 2000, the department of Commerce Bureau of Export revised encryption export regulations. The results of the new export regulations it that the regulations have been greatly relaxed. However if the code is to be exported outside of the source country current export laws for the export and import counties should be reviewed for compliance.

Case in point is if the entire message is hashed instead of a digital signature of the of message the National Security Agency (NSA) considers this a quasi-encryption and State controls would apply.

It is always a valid choice to seek legal advice within the organization that the code review is being done to ensure legal compliance.

Lifecycle
With security nothing is secure forever. This is especially true with cryptographic hashing functions.  Some hashing algorithms such as Windows LanMan hashes are considered completely broken. The code reviewer needs to understand the weaknesses of obsolete hashing functions as well as the current best practices for the choice of cryptographic algorithms.

Programming/Vulnerabilities
The most common programmatic issue with hashing is not using a salt value or if using a salt the salt value is too short and or the same salt value is used in multiple hashes. The purpose of a salt is to make it harder for an attacker to perform pre-computed hashing attack (e.g., using rainbow tables) but other benefits of a salt can include making it difficult for an attacker to perform even password guessing attacks by obfuscating the hashed value.

Salt
One way to generate a secure salt value is using a pseudo-random number generator. Note that a salt value does not need to possess the quality of a cryptographically secure randomness.
Best practices is to use a cryptographically function to create the salt, salt value should be created for each hash value, and a minimum value of 128 bits. The bits are not costly so don't save a few bits thinking you gain something back in performance instead use a value of 256-bit salt value. It is highly recommended.

.Net Salt
    private int minSaltSize = 8;
    private int maxSaltSize = 24;
    private int saltSize;
 
    private byte[] GetSalt(string input) {
            byte[] data;
            byte[] saltBytes;
            RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
            saltBytes = new byte[saltSize];
            rng.GetNonZeroBytes(saltBytes);
            data = Encoding.UTF8.GetBytes(input);
            byte[] dataWithSaltBytes =
                new byte[data.Length + saltBytes.Length];
            for (int i = 0; i < data.Length; i++)
                dataWithSaltBytes[i] = data[i];
            for (int i = 0; i < saltBytes.Length; i++)
                dataWithSaltBytes[data.Length + i] = saltBytes[i];
            return dataWithSaltBytes;
        }

This method uses an agile approach to calling a hash function. It is explained below.

     private string computeHashWithSalt(HashAlgorithm myHash, string input) {
            byte[] data;
            data = myHash.ComputeHash(GetSalt(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }

Microsoft .Net Notes on Hashing
Microsoft does not recommend using MD5 or SHA-1. With .Net 3.5 and above Microsoft supports the Suite B set of cryptographic algorithms published by the National Security Agency (NSA).

The salt value does not need to be secret and can be stored along with the hash value. Some may use a combination of account details (username, user full name, ID, creation date, etc.) as the salt for hash to further obfuscate the hash computation: for example salt = (username|lastname|firstname|ID|generated_salt_value).

Best Practices
Industry leading Cryptographer’s are advising that MD5 and SHA-1 should not be used for any applications. The United State FEDERAL INFORMATION PROCESSING STANDARDS PUBLICATION (FIPS) specifies seven cryptographic hash algorithms — SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, SHA-512/224, and SHA-512/256 are approved for federal use. The code reviewer should consider this standard because the FIPS is also widely adopted by the information technology industry.

The code reviewer should raise a red flag if MD5 and SHA-1 are used and a risk assessment be done to understand why these functions would be used instead of other better-suited hash functions. FIPS does allow that MD5 can be used only when used as part of an approved key transport scheme (e.g. SSL v3.1) where no security is provided by the algorithm.

FIPS disapproves the following functions DES; MD51; RC4; Blowfish; Diffie-Hellman2; Diffie-Hellman3 (key agreement); EC Diffie-Hellman2 (key agreement); AES4 (non-compliant); Diffie-Hellman5 (key agreement); EC Diffie-Hellman4 (vendor affirmed); RSA4 (key agreement); RSA2 (key wrapping).

.Net Agile Code example for hashing
App Code File:
<add key="HashMethod" value="SHA512"/>

C# Code:
   1:  preferredHash = HashAlgorithm.Create((string)ConfigurationManager.AppSettings["HashMethod"]);
   2:  
   3:  hash = computeHash(preferredHash, testString);
   4:  
   5:  private string computeHash(HashAlgorithm myHash, string input) {
   6:       byte[] data;
   7:       data = myHash.ComputeHash(Encoding.UTF8.GetBytes(input));
   8:       sb = new StringBuilder();
   9:       for (int i = 0; i < data.Length; i++) {
  10:           sb.Append(data[i].ToString("x2"));
  11:       }
  12:      return sb.ToString();
  13:  }


Line 1 let's us get our hashing algorithm we are going to use from the config file. If we use the machine config file our implementation would be server wide instead of application specific.
Line 3 allows us to use the config value and set it according as our choice of hashing function. ComputHash can be SHA-256 or SHA-512.

The drawback to this method is key size. I would suggest of giving yourself twice the size of the largest key of hashing algorithm you could possible use to store hash values. This means we need a varchar of 1024 if we are going to store our hash value in the database.

Afterword
Lastly, never accept in a code review an algorithm created by the programmer for hashing or copy a hashing function taken from the Internet. Always use cryptographic functions that are provided by the language framework the code is written in. These functions are well vetted and well tested by experience cryptographers.



References:
* http://valerieaurora.org/hash.html  (Lifetimes of cryptographic hash functions)
* http://msdn.microsoft.com/en-us/library/system.security.cryptography.rngcryptoserviceprovider.aspx
* http://csrc.nist.gov/publications/fips/fips180-4/fips-180-4.pdf
* Ferguson and Schneier (2003) Practical Cryptography (see Chapter 6; section 6.2 Real Hash Functions)



Saturday, June 23, 2012

.Net Agile Cryptographic Function

A lot has recently been written about the recent LinkedIn hack. I am not going to delve into the details of the hack but I do want to show how you can have an agile cryptographic function for creating hash values. By agile I mean that the code we can use to create our hash value is not hard coded within the program. 
 MD5 is the weakest hashing algorithm we have in the .Net framework. MD5 might be strong enough when our first application was written but now we want to implement a stronger hashing function. We can re-code our application to use windows default hashing function. SHA-1 is the current default hashing algorithm but SHA-1 still doesn’t provide us with the strongest algorithm we can use. So now our boss has told us about the Linkedin hack and has asked us to use SHA512. Using an agile approach we can say “Ok, I can have that done in about 1 hour” without deploying any code. All we have to do is change our app or machine confg file. 

 We can accomplish this by using the following lines of code.


App Code File:



   <add key="HashMethod" value="SHA512"/>

C# Code:

1: preferredHash = HashAlgorithm.Create((string)ConfigurationManager.AppSettings["HashMethod"]);

   2:   
   3:  hash = computeHash(preferredHash, testString);
   4:   
   5:  private string computeHash(HashAlgorithm myHash, string input) {
   6:       byte[] data;
   7:       data = myHash.ComputeHash(Encoding.UTF8.GetBytes(input));
   8:       sb = new StringBuilder();
   9:       for (int i = 0; i < data.Length; i++) {
  10:           sb.Append(data[i].ToString("x2"));
  11:       }
  12:      return sb.ToString();
  13:  }



Line 1 let's us get our hashing algorithm we are going to use from the config file. If we use the machine config file our implementation would be server wide instead of application specific. 

The drawback to this method is key size. I would suggest of giving yourself twice the size of the largest key of hashing algorithm  you could possible use to store hash values. This means we need a varchar of 1024 if we are going to store our hash value in the database.

I will leave it up to the reader to take the hash values through Rainbow tables to see how quickly Raindow tables can return the value of the hash. 

Demo Code:

using System;
using System.Text;
using System.Security.Cryptography;
using System.Configuration;
using System.Collections;

namespace HashDemo {
    class HashTest {

        private MD5 md5Hash;
        private HashAlgorithm cryptoDefaultHash;
        private HashAlgorithm preferredHash;
        private Random random;
        private string testString = string.Empty;
        private string hash;
        private StringBuilder sb;
        private int minSaltSize = 8;
        private int maxSaltSize = 24;
        private int saltSize;

        public HashTest() {
            testString = "MySup3r#Secur3&Pa()sw0rd*";  // My Super Secure Password

            md5Hash = MD5.Create();  // Application hard codes Hash Algorithm. Not good must change code to use a newer Crypto Algorithm.

            cryptoDefaultHash = HashAlgorithm.Create(); // A little better, but uses windows default SHA-1 Crypto Algorithm, but SHA-1 is weak
            ///
            preferredHash = HashAlgorithm.Create((string)ConfigurationManager.AppSettings["HashMethod"]); // Best way is to use app or machine config file to set preferred Crypto Algorithm.
            ///
            Console.WriteLine("The crypto function is: " + md5Hash.GetType() + ".");
            Console.WriteLine("The crypto function is: " + cryptoDefaultHash.GetType() + ".");
            Console.WriteLine("The crypto function is: " + preferredHash.GetType() + ".");
            random = new Random();
            saltSize = random.Next(minSaltSize, maxSaltSize);
        }

        public void test() {
            hash = computeMd5Hash(testString);
            Console.WriteLine("MD5 - The hash is: " + hash + ".");
            hash = computeMd5HashWithSalt(testString);
            Console.WriteLine("MD5 with Salt - The hash is: " + hash + ".");

            hash = computeDefaultHash(cryptoDefaultHash, testString);
            Console.WriteLine("Default hash is: " + hash + ".");
            hash = computeDefaultHashWithSalt(cryptoDefaultHash, testString);
            Console.WriteLine("Default hash with Salt is: " + hash + ".");

            hash = computeHash(preferredHash, testString);
            Console.WriteLine("Preferred hash with Salt is: " + hash + ".");
            hash = computeHashWithSalt(preferredHash, testString);
            Console.WriteLine("Preferred hash with Salt is: " + hash + ".");
        }

        private string computeMd5Hash(string input) {
            byte[] data;
            data = md5Hash.ComputeHash(Encoding.UTF8.GetBytes(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }
        private string computeMd5HashWithSalt(string input) {
            byte[] data;
            data = md5Hash.ComputeHash(GetSalt(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }
        private string computeDefaultHash(HashAlgorithm myHash, string input) {
            byte[] data;
            data = myHash.ComputeHash(Encoding.UTF8.GetBytes(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }
        private string computeDefaultHashWithSalt(HashAlgorithm myHash, string input) {
            byte[] data;
            data = myHash.ComputeHash(GetSalt(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }
        private string computeHash(HashAlgorithm myHash, string input) {
            byte[] data;
            data = myHash.ComputeHash(Encoding.UTF8.GetBytes(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }
        private string computeHashWithSalt(HashAlgorithm myHash, string input) {
            byte[] data;
            data = myHash.ComputeHash(GetSalt(input));
            sb = new StringBuilder();
            for (int i = 0; i < data.Length; i++) {
                sb.Append(data[i].ToString("x2"));
            }
            return sb.ToString();
        }

        private byte[] GetSalt(string input) {
            byte[] data;
            byte[] saltBytes;
            RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
            saltBytes = new byte[saltSize];
            rng.GetNonZeroBytes(saltBytes);
            data = Encoding.UTF8.GetBytes(input);
            byte[] dataWithSaltBytes =
                new byte[data.Length + saltBytes.Length];
            for (int i = 0; i < data.Length; i++)
                dataWithSaltBytes[i] = data[i];
            for (int i = 0; i < saltBytes.Length; i++)
                dataWithSaltBytes[data.Length + i] = saltBytes[i];
            return dataWithSaltBytes;
        }

        ~HashTest() { // dtor
            preferredHash.Clear();
            cryptoDefaultHash.Clear();
            md5Hash.Clear();
        }
    } // end of class
} // end of namespace

Program Output:

The crypto function is: System.Security.Cryptography.MD5CryptoServiceProvider.
The crypto function is: System.Security.Cryptography.SHA1CryptoServiceProvider.
The crypto function is: System.Security.Cryptography.SHA512Managed.

MD5 - The hash is: 6c27f9f54f905b815e01298ccef39420.
MD5 with Salt - The hash is: 60148ef74c3dbce6ab11c5897f110167.

Default hash is: 636f54fe86c4518308f2599e5f5d8adc72cdc79c.
Default hash with Salt is: 2b223ced9cb488dfe842f3acc32c192c14f0606b.

Preferred hash with Salt is: 48ccbc77e2f280e277ebd871af9871b624c915559f677ba5c16
2d9156e70c46aa5fcc539dec4cc51155101c4e638d226746cc4a7b821bbc5501e104fbeee5eb3.
Preferred hash with Salt is: 6cd578abf2d9b65d621d42fb23e57aede34b401507a4d57d3a7
00d2dbd4f2376b4632a3add3e44c71249e1ed5a4ee293ddfe45efd9e2d0102909d10b74b64483.