When is paranoia a good thing?
When it comes to downloading files for sure. Using a cryptographic hash function to test the authenticity of a file is a good practice. Such a good practice, that I wanted to make it as easy as possible to perform.
I’ve written two scripts – one in PowerShell and the other in Bash shell – they do the same thing. Given a filename to create a SHA-512 hash for, both generate a hash value and compare it to one provided by the file owner, and then tell you whether they match (or warns you if they don’t).
If you use a lot of Open Source software you should get in the habit of making sure you’re receiving what you expected. It’s not unusual to find that very large files have suffered corruption (for example, by terminating before completely downloading) – testing the received file’s hash is a good way to make sure you have the complete file. I’ve decided to expect to compare a SHA512 hash, only because I find it to be quite common. And because I needed something to script. Both of these scripts could easily be changed to make use of your preferred hash function. I’m going to assume that there’s nothing malicious about the file I’ve download, at least in terms of the author, however I want to use the hash function to confirm that no third party has intercepted and replaced the file with something that I shouldn’t open.
The point of this exercise was threefold:
1/ It’s a good discipline to ensure you’re receiving what you expected,
2/ I wanted to compare how easy it is to code with PowerShell against how easy it is to achieve the same thing with Bash shell, and,
3/ my intention was to create a kind of Rosetta stone for myself (and hopefully others) – these trivial scripts contain fairly representative examples of the most commons operations that appear in scripts (IMHO). I don’t have to write PowerShell too often, at present, so I wanted to keep a reference handy for the next time I need to do something. Similarly with Bash – I don’t write scripts everyday, and so it’s useful to remind myself how finicky it is about things like spaces, and evaluating commands etc.
If you’re just here to get a script to test the hash value of a file you’ve downloaded for Windows or Unix (Linux or macOS/osX) – please feel free to use the appropriate script from here. If you’re wondering how to retrieve parameters from the command line, test if variable has a value, read user input, make sure a file exists, convert the case of a string, trim some part of a string or compare two strings – you should be able to find the answer here too.
Lessons:
Comparing the two scripting languages: I’m still a big Bash fan, so I guess I was never going to start saying glowing things about PowerShell, it works OK. To be fair, I’ve never tried to read a book on PowerShell (I probably should) – and so I mostly hack my way through it and shouldn’t be too critical. There’s a small glow of satisfaction every time I manage to make it doing something I couldn’t do before. The two scripts are pretty much the same length – no surprises there, but I find it encouraging that it’s about the same effort in both languages.
I didn’t know how pass parameters from the command line to PowerShell and now I do. I struggled for a wee-while on how to test if the variable was empty, that wasn’t as obvious as I’d hoped – but now I’ve got a pattern for it.
With the Bash script, I hadn’t used “getopts” before – combining it with a case statement turned out to be quite an elegant way of dealing with the parameters – especially unexpected ones. It has the advantage of being easy to maintain. I always forget that Bash is extremely finicky about spaces and square brackets (especially) in conditional statements – it might stick this time. (BTW: Don’t forget, if you need to debug your Bash you just add “-x” to the end of the first line and you’ll get plenty of useful info on what’s happening. With PowerShell, in Windows10, you’d run “powershell_ise.exe” and open your script for debugging to step through it.)
If you have a better way (in terms of script writing, rather the checking hashes), please feel free to comment.
PowerShell Example:
Download check_SHA512.pw1 from GitLabs
# Script Name: check_SHA512.ps1
# Author : Nick
# Date: 16 July 2019
# Purpose: Quickly compare the SHA512 calculated hash of downloaded
# file with the hash provided by another party (the author hopefully).
# Usage: The script will prompt for the name of a downloaded
# this can include the path and for the hash value provided by the
# source file. The script will confirm whether they agree or otherwise.
# Command line usage:
# .\check_SHA512.ps1 -file myFile.txt -hash 3dff9bc9283d34a983b…
# Warning: This script only confirms that you have received what a remote
# site considers to be a true copy of a file, you should still treat the file
# with suspicion (eg malware or virus risk) if you do not know the author.
Param (
[string]$file = $( Read-Host "Enter the filename you've received" ),
[string]$hash = $( Read-Host "Enter the hash value you copied from the remote source" )
)
# Just wanted to manage the parameters with more meaningful names
if (-not ([string]::IsNullOrEmpty($file))) {
$receivedFilename = $file
}
if (-not ([string]::IsNullOrEmpty($hash))) {
$remoteHash = $hash
}
try {
$calculatedHash = ((Get-FileHash $receivedFilename -Algorithm SHA512 -ErrorAction Stop) | Select -Expand "Hash" )
}
catch {
Write-Host "There was an error calculating the hash for the file you specified.`nCheck the file exists and try again."
break
}
# Strings in PowerShell are case insensitive, this is just for formatting.
$calculatedHash = $calculatedHash.ToUpper()
$remoteHash = $remoteHash.ToUpper()
if ($calculatedHash -eq $remoteHash) {
Write-Host "The hashes agree, the downloaded file appears to be the same as the source.`nNow scan the file for malware and viruses."
} else {
Write-Host "The hashes did NOT agree. Do NOT open the file until you've resolved why."
Write-Host "The remote hash was : $remoteHash"
Write-Host "The local hash was : $calculatedHash"
}
Note(s): If you decide to download this file you’ll be warned that it poses a security risk. If you’re not completely comfortable with what it does, I suggest you don’t use it. Each time you execute it Windows will continue to give you security warnings. You can “trust” the script (and stop receiving the warnings) by executing the PowerShell command:
Unblock-File .\check_SHA512.ps1
This PowerShell script was developed using version 5.1, this may prove useful if the script reports unexpected errors. You can confirm your version (major and minor version) by running the command:
$PSVersionTable.PSVersion
This reported the following:
Major Minor Build Revision
----- ----- ----- --------
5 1 17763 316
Bash shell Example:
Download check_SHA512.sh from GitLabs
#!/bin/bash
# Script Name: check_SHA512.sh
# Author: Nick
# Date Written: 18 June 2019
# Purpose: Compare the checksum of a downloaded file against a received value.
#
# Usage: The script can be invoked using parameters, but if it doesn't receive them
# it will prompt for what it requires.
#
# Command line usage:
# ./check_SHA512.sh -f myFile.txt -h 3dff9bc9283d34a983b...
#
# Warning: This script only confirms that you have received what a remote site
# considers to be a true copy of a file, you should still treat the file with
# suspicion (eg malware or virus risk) if you do not know the author.
#
# Test the parameters, if we get more than expected then stop
if [ $# -gt 4 ]; then
echo -e "Unexpected parameters\n"
echo "Expected:"
echo "./check_hash.sh -f myFile.txt -h 3dff9bc9283d34a983b..."
echo "If you call ./check_hash.sh without parameters it will prompt you for what it needs"
exit
else
while getopts f:h: option
do
case "$option"
in
f) filename=${OPTARG};;
h) remoteHash=${OPTARG};;
*) echo "Unexpected argument"; exit;;
esac
done
fi
if [ -z "$filename" ]; then
read -p "Enter the filename you've received: " filename
fi
if [ -z "$remoteHash" ]; then
read -p "Enter the SHA512 has you received: " remoteHash
fi
# Now that we have a filename, let's make sure it exists, and quit if it doesn't
# And if it exists, let's make sure it's not a directory
if [ ! -f "$filename" ]; then
echo "Couldn't find a file named: $filename"
exit
fi
# Ok - we can now see what hash we get from the file, using sed to just get the hash
calculatedHash=$(openssl sha -sha512 $filename | sed 's/^.* //')
# Let's make sure bash our hash and the received hash are upper case before comparing
calculatedHash=$(echo $calculatedHash | tr a-z A-Z)
remoteHash=$(echo $remoteHash | tr a-z A-Z)
# Finally, we can compare the hashes and give a result
if [ "$calculatedHash" = "$remoteHash" ]; then
echo -e "The hashes agree, the downloaded file appears to be the same as the source.\nNow scan the file for malware and viruses."
else
echo "The hashes did NOT agree. Do NOT open the file until you've resolved why."
echo "The remote hash was: $remoteHash"
echo "The local hash was: $calculatedHash"
fi
# Throw in an extra newline to make it easier to see this has completed
echo
exit
Note: To be able to run ‘check_SHA512.sh’ you’ll need to give it ‘execute’ permission:
chmod u+x ./check_SHA512.sh
Finally as a way of confirming that it does what it says it does, you can execute either script to make it confirm that it’s the version that you expected. I have done this using the files stored on GitLabs – cutting and pasting the text to create the files may result in a different hash being reported. To test either of the two downloadable scripts, run either:
PowerShell:
./check_SHA512.ps1 check_SHA512.ps1 410DFB6BF838D3688018C3C101B2AAA2AD2FE440DDF4E620FD8377A8FB0154
F6ECD736B637AF08FE558C6C0F405013A542F2718618414A652912F08F7A6C3EA6
Bash:
./check_SHA512.sh check_SHA512.sh AFFB1B0D28EAD107714E930AAFC08D65F0FA9FA347188E9620CF1D110E91FE25F
006C317C60252232CB8A1D176DB8D79507B4BB8A807247010925EBAB655F07B>
Slàinte mhath, Nick.
