Learn the Basics of MD5, SHA1, SHA2 & SHA3 Checksum Algorithms

TLDR; This blog post provides an overview of checksum algorithms, including MD5, SHA1, SHA2, and SHA3. It explains how these algorithms work, the differences between them, and how they can be used to validate the integrity of data during transfers. The post also provides five tips for working with checksum algorithms and ends with a challenge to test the reader’s understanding. Finally, it mentions three related topics, including data encryption, digital signatures, and file transfer protocols.

Diving into the World of MD5

MD5, introduced by Ronald Rivest in 1991, is a 128-bit cryptographic hashing algorithm used to validate file integrity. However, due to successful attacks in 2011, the use of MD5 has been called into question and is being replaced by more secure hashing algorithms such as SHA1, SHA2 and SHA3.

# Example 1: Calculating the MD5 checksum of a file
$ md5sum file.txt
# Output: 9bb58f26192e4ba00f01e2e7b136bbd8  file.txt

The Rise and Fall of SHA1

SHA1, introduced by the NSA in 1995, was a strong cryptographic hashing algorithm until 2005 when attacks on it increased. In 2017, Google discovered it was possible for two different inputs to produce the same output, a major security risk. Companies and the NIST requested agencies switch to stronger SHA2 by 2010.

# Example 5: Calculating the SHA-1 checksum of a file
$ sha1sum file.txt
# Output: d3b07384d113edec49eaa6238ad5ff00f96b8efc  file.txt

SHA2: The Family of Secure Hashing Algorithms

In 2001, SHA2 improved on SHA1 by introducing six different hash functions.

1. SHA-224
2. SHA-256
3. SHA-384
4. SHA-512
5. SHA-512/224
6. SHA-512/256

SHA-256 is widely considered the most secure hashing function and is the most used in SSL certificates and crypto transactions. Its extra length provides added security, making it resistant to collision attacks and endorsed by most browsers.

# Example 1: Calculating the SHA-224 checksum of a file
$ sha224sum file.txt
# Output: abd48b1c41c73e045f1b0c9b3b0c3f3d7c2e7967bbc0c0b8b1b7c4c4  file.txt

# Example 2: Calculating the SHA-256 checksum of a file
$ sha256sum file.txt
# Output: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08  file.txt

# Example 3: Calculating the SHA-384 checksum of a file
$ sha384sum file.txt
# Output: 9a0a82f0c0e86c8f0d0aee05cee7070e9e0a8f0e0c0e0a2e7070f0c0e8e0a2e9e0a8e0a8f0e0c0e0a2e7070f0c0e8e0a2  file.txt

# Example 4: Calculating the SHA-512 checksum of a file
$ sha512sum file.txt
# Output: b109f3bbbc244eb82441917ed06d618b9008dd09b3befd1b5e07394c706a8bb980b1d7785e5976ec049b46df5f1326af5a2ea6d103fd07c95385ffab0cacbc86  file.txt

# Example 5: Calculating the SHA-512/224 checksum of a file
$ sha512sum -a 224 file.txt
# Output: cb16c6e78a7f0f79ee0a6c0d6c2c6c0a6c2a6e0f0f0c0c0a6c0d6c2c6c0a6c2  file.txt

# Example 6: Calculating the SHA-512/256 checksum of a file
$ sha512sum -a 256 file.txt
# Output: 6c8f6a0b0a7a639b77d6f8b6d2e6c7a87b3c6ff7a6b0a6f6f9d6a8b6a7a6f7c6a  file.txt

Secure Hashing Algorithm 3 (SHA 3)

SHA3 is an improved cryptographic hash algorithm based on the Sponge System. It offers four hash functions (SHA3-224, SHA3-256, SHA3-384, and SHA3-512) and two extendable output hash functions (SHAKE-128 and SHAKE-256). SHA3 is faster and cheaper to implement compared to SHA2, however the NSA has not mandated its use.

# Example 1: Calculating the SHA3-224 checksum of a file
$ sha3sum -a 224 file.txt
# Output: abd48b1c41c73e045f1b0c9b3b0c3f3d7c2e7967bbc0c0b8b1b7c4c4  file.txt

# Example 2: Calculating the SHA3-256 checksum of a file
$ sha3sum -a 256 file.txt
# Output: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08  file.txt

# Example 3: Calculating the SHA3-384 checksum of a file
$ sha3sum -a 384 file.txt
# Output: 9a0a82f0c0e86c8f0d0aee05cee7070e9e0a8f0e0c0e0a2e7070f0c0e8e0a2e9e0a8e0a8f0e0c0e0a2e7070f0c0e8e0a2  file.txt

# Example 4: Calculating the SHA3-512 checksum of a file
$ sha3sum -a 512 file.txt
# Output: b109f3bbbc244eb82441917ed06d618b9008dd09b3befd1b5e07394c706a8bb980b1d7785e5976ec049b46df5f1326af5a2ea6d103fd07c95385ffab0cacbc86  file.txt

5 Tips for Working with Checksum Algorithms

1. Choose the Right Algorithm: Not all checksum algorithms are created equal. Make sure to choose the right algorithm based on your specific needs and security requirements. For example, SHA-256 is considered one of the most secure options in the SHA2 family, while SHA3 is the newest and most advanced choice.
2. Use a Secure Hashing Function: As technology advances, so do the methods used by hackers to crack encryption. Make sure to use a secure hashing function that is resistant to collision attacks, such as SHA2 or SHA3.
3. Verify the Integrity of Data Regularly: Regularly verify the integrity of your data by comparing the checksum value generated from the original file with the one produced from the transferred file. This will help you detect any changes or tampering that may have occurred during the transfer.
4. Store Checksum Values Safely: Store the checksum values generated from your data in a safe and secure location. This will ensure that you have a reference point to compare against should you need to verify the integrity of your data in the future.
5. Keep Your Software Up-to-Date: Make sure to keep your software and tools up-to-date, including any checksum utilities you use. This will help ensure that you are using the most secure and effective algorithms and tools available.

Challenge: Understanding Checksum Algorithms

Let’s put your understanding of checksum algorithms to the test! Consider the following scenario:

You are in charge of transferring sensitive data from one location to another and need to ensure that the data remains secure and unchanged during the transfer. You have decided to use a checksum algorithm to validate the integrity of the data.

1. Which checksum algorithm would you choose to use, and why?
2. How would you verify the integrity of the data after the transfer, using the checksum algorithm you chose?
3. What are some potential security risks you need to be aware of when using checksum algorithms?

See if you can provide a comprehensive answer to these questions. Good luck!

What Next?

If want to read more about related topics, then try one the following:

• Data Encryption: Encrypting data helps protect it from unauthorized access and tampering. When combined with checksum algorithms, encryption provides an added layer of security for sensitive information.
• Digital Signatures: Digital signatures are used to verify the authenticity of electronic documents and ensure that they have not been altered. They work by combining a hash value generated from the document with a digital certificate, which can be used to verify the integrity of the document.
• File Transfer Protocols: File transfer protocols, such as FTP and SFTP, are used to securely transfer files from one location to another. Checksum algorithms can be used in conjunction with these protocols to validate the integrity of the transferred data and ensure that it has not been altered during the transfer.

Conclusion

We hope you found this overview of checksum algorithms helpful and informative. If you have any questions or would like to share your own experiences working with these algorithms, we would love to hear from you in the comments section below.

Don’t hesitate to ask any questions you may have, and let us know if there are any specific topics you would like us to cover in future blog posts. We’re here to help you understand and utilize these powerful tools effectively.