enh: factorize file download func

FAQ.md

@@ -45,9 +45,9 @@ Software vulnerability:
 
 Hardware vulnerability:
 - Can be fixed? No, only mitigated (or buy new hardware!)
-- How to ~~fix~~ mitigate? In the worst case scenario, 5 "layers" need to be updated: the microcode/firmware, the host OS kernel, the hypervisor, the VM OS kernel, and possibly all the software running on the VM.
+- How to ~~fix~~ mitigate? In the worst case scenario, 5 "layers" need to be updated: the microcode/firmware, the host OS kernel, the hypervisor, the VM OS kernel, and possibly all the software running on the machine. Sometimes only a subset of those layers need to be updated. In yet other cases, there can be several possible mitigations for the same vulnerability, implying different layers. Yes, it can get horribly complicated.
 
-A more detailed video explanation is available here: https://youtu.be/2gB9U1EcCss?t=85
+A more detailed video explanation is available here: https://youtu.be/2gB9U1EcCss?t=425
 
 ## What do "affected", "vulnerable" and "mitigated" mean exactly?
 
@@ -75,7 +75,7 @@ There are a few rules that govern how this tool is written.
 
 A lot as changed since 2018. Nowadays, the industry adapted and this range of vulnerabilities is almost "business as usual", as software vulnerabilities are. However, due to their complexity, it's still not as easy as just checking a version number to ensure a vulnerability is closed.
 
-Granted, we now have a standard way under Linux to check whether our system is affected, vulnerable, mitigated against most of these vulnerabilities. By having a look at the `sysfs` hierarchy, and more precisely the `/sys/devices/system/cpu/vulnerabilities/` folder, one can have a pretty good insight about its system state for each of the listed vulnerabilities. Note that the output can be a little different with some vendors (e.g. Red Hat has some slightly different output than the vanilla kernel for some vulnerabilities), but it's still a gigantic leap forward, given where we were in 2018 when this script was started, and it's very good news. The kernel is the proper place to have this because the kernel knows everything about itself (the mitigations it might have), and the CPU (its model, and microcode features that are exposed).
+Granted, we now have a standard way under Linux to check whether our system is affected, vulnerable, mitigated against most of these vulnerabilities. By having a look at the `sysfs` hierarchy, and more precisely the `/sys/devices/system/cpu/vulnerabilities/` folder, one can have a pretty good insight about its system state for each of the listed vulnerabilities. Note that the output can be a little different with some vendors (e.g. Red Hat has some slightly different output than the vanilla kernel for some vulnerabilities), but it's still a gigantic leap forward, given where we were in 2018 when this script was started, and it's very good news. The kernel is the proper place to have this because the kernel knows everything about itself (the mitigations it might have), and the CPU (its model, and microcode features that are exposed). Note however that some vulnerabilities are not reported through this file hierarchy at all, such as Zenbleed.
 
 However I see a few reasons why this script might still be useful to you, and that's why its development has not halted when the `sysfs` hierarchy came out:
 
@@ -109,12 +109,13 @@ This tool only supports Linux, and [some flavors of BSD](#which-bsd-oses-are-sup
 
 ## The tool says there is an updated microcode for my CPU, but I don't have it!
 
-Even if your operating system is fully up to date, the tool might still tell you that there is a more recent microcode version for your CPU. Currently, it uses (and merges) information from two sources:
+Even if your operating system is fully up to date, the tool might still tell you that there is a more recent microcode version for your CPU. Currently, it uses (and merges) information from three sources:
 
 - The official [Intel microcode repository](https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files)
 - The awesome platomav's [MCExtractor database](https://github.com/platomav/MCExtractor) for non-Intel CPUs
+- The official [linux-firmware](https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git) repository for AMD
 
-Generally, for Intel CPUs it means that Intel does have a more recent version for your CPU, and for other CPUs it means that a more recent version has already been seen in the wild. However, your OS vendor might have chosen not to ship this new version (yet), maybe because it's currently being tested, or for other reasons. This tool can't tell you when or if this will be the case. You should ask your vendor about it. Technically, you can still go and upgrade your microcode yourself, and use this tool to confirm whether you did it successfully. Updating the microcode for you is out of the scope of this tool, as this would violate [rule 1b](#what-are-the-main-design-decisions-regarding-this-script).
+Generally, it means a more recent version of the microcode has been seen in the wild. However, your OS vendor might have chosen not to ship this new version (yet), maybe because it's currently being tested, or for other reasons. This tool can't tell you when or if this will be the case. You should ask your vendor about it. Technically, you can still go and upgrade your microcode yourself, and use this tool to confirm whether you did it successfully. Updating the microcode for you is out of the scope of this tool, as this would violate [rule 1b](#what-are-the-main-design-decisions-regarding-this-script).
 
 ## The tool says that I need a more up-to-date microcode, but I have the more recent version!
 

spectre-meltdown-checker.sh

@@ -118,24 +118,27 @@ show_disclaimer()
 Disclaimer:
 
 This tool does its best to determine whether your system is immune (or has proper mitigations in place) for the
-collectively named "speculative execution" vulnerabilities. It doesn't attempt to run any kind of exploit, and can't guarantee
+collectively named "transient execution" (aka "speculative execution") vulnerabilities that started to appear
-that your system is secure, but rather helps you verifying whether your system has the known correct mitigations in place.
+since early 2018 with the infamous Spectre & Meltdown.
+
+This tool does NOT attempt to run any kind of exploit, and can't 100% guarantee that your system is secure,
+but rather helps you verifying whether your system has the known correct mitigations in place.
 However, some mitigations could also exist in your kernel that this script doesn't know (yet) how to detect, or it might
 falsely detect mitigations that in the end don't work as expected (for example, on backported or modified kernels).
 
-Your system exposure also depends on your CPU. As of now, AMD and ARM processors are marked as immune to some or all of these
+Your system affectability to a given vulnerability depends on your CPU model and CPU microcode version, whereas the
-vulnerabilities (except some specific ARM models). All Intel processors manufactured since circa 1995 are thought to be vulnerable,
+mitigations in place depend on your CPU (model and microcode), your kernel version, and both the runtime configuration
-except some specific/old models, such as some early Atoms. Whatever processor one uses, one might seek more information
+of your CPU (through bits set through the MSRs) and your kernel. The script attempts to explain everything for each
-from the manufacturer of that processor and/or of the device in which it runs.
+vulnerability, so you know where your system stands. For a given vulnerability, detailed information is sometimes
+available using the \`--explain\` switch.
 
-The nature of the discovered vulnerabilities being quite new, the landscape of vulnerable processors can be expected
+Please also note that for the Spectre-like vulnerabilities, all software can possibly be exploited, in which case
-to change over time, which is why this script makes the assumption that all CPUs are vulnerable, except if the manufacturer
+this tool only verifies that the kernel (which is the core of the system) you're using has the proper protections
-explicitly stated otherwise in a verifiable public announcement.
+in place. Verifying all the other software is out of the scope of this tool, as it can't be done in a simple way.
+As a general measure, ensure you always have the most up to date stable versions of all the software you use,
+especially for those who are exposed to the world, such as network daemons and browsers.
 
-Please also note that for Spectre vulnerabilities, all software can possibly be exploited, this tool only verifies that the
+For more information and answers to related questions, please refer to the FAQ.md file.
-kernel (which is the core of the system) you're using has the proper protections in place. Verifying all the other software
-is out of the scope of this tool. As a general measure, ensure you always have the most up to date stable versions of all
-the software you use, especially for those who are exposed to the world, such as network daemons and browsers.
 
 This tool has been released in the hope that it'll be useful, but don't use it to jump to conclusions about your security.
 
@@ -881,6 +884,29 @@ fms2cpuid()
 	echo $(( (_stepping & 0x0F) | (_lowmodel << 4) | (_lowfamily << 8) | (_extmodel << 16) | (_extfamily << 20) ))
 }
 
+download_file()
+{
+	_url="$1"
+	_file="$2"
+	if command -v wget >/dev/null 2>&1; then
+		wget -q "$_url" -O "$_file"; ret=$?
+	elif command -v curl >/dev/null 2>&1; then
+		curl -sL "$_url" -o "$_file"; ret=$?
+	elif command -v fetch >/dev/null 2>&1; then
+		fetch -q "$_url" -o "$_file"; ret=$?
+	else
+		echo ERROR "please install one of \`wget\`, \`curl\` of \`fetch\` programs"
+		unset _file _url
+		return 1
+	fi
+	unset _file _url
+	if [ "$ret" != 0 ]; then
+		echo ERROR "error $ret"
+		return $ret
+	fi
+	echo DONE
+}
+
 [ -z "$HOME" ] && HOME="$(getent passwd "$(whoami)" | cut -d: -f6)"
 mcedb_cache="$HOME/.mcedb"
 update_fwdb()
@@ -897,42 +923,14 @@ update_fwdb()
 	mcedb_tmp="$(mktemp -t smc-mcedb-XXXXXX)"
 	mcedb_url='https://github.com/platomav/MCExtractor/raw/master/MCE.db'
 	_info_nol "Fetching MCE.db from the MCExtractor project... "
-	if command -v wget >/dev/null 2>&1; then
+	download_file "$mcedb_url" "$mcedb_tmp" || return $?
-		wget -q "$mcedb_url" -O "$mcedb_tmp"; ret=$?
-	elif command -v curl >/dev/null 2>&1; then
-		curl -sL "$mcedb_url" -o "$mcedb_tmp"; ret=$?
-	elif command -v fetch >/dev/null 2>&1; then
-		fetch -q "$mcedb_url" -o "$mcedb_tmp"; ret=$?
-	else
-		echo ERROR "please install one of \`wget\`, \`curl\` of \`fetch\` programs"
-		return 1
-	fi
-	if [ "$ret" != 0 ]; then
-		echo ERROR "error $ret while downloading MCE.db"
-		return $ret
-	fi
-	echo DONE
 
 	# second, get the Intel firmwares from GitHub
 	intel_tmp="$(mktemp -d -t smc-intelfw-XXXXXX)"
 	intel_url="https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/archive/main.zip"
 	_info_nol "Fetching Intel firmwares... "
 	## https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files.git
-	if command -v wget >/dev/null 2>&1; then
+	download_file "$intel_url" "$intel_tmp/fw.zip" || return $?
-		wget -q "$intel_url" -O "$intel_tmp/fw.zip"; ret=$?
-	elif command -v curl >/dev/null 2>&1; then
-		curl -sL "$intel_url" -o "$intel_tmp/fw.zip"; ret=$?
-	elif command -v fetch >/dev/null 2>&1; then
-		fetch -q "$intel_url" -o "$intel_tmp/fw.zip"; ret=$?
-	else
-		echo ERROR "please install one of \`wget\`, \`curl\` of \`fetch\` programs"
-		return 1
-	fi
-	if [ "$ret" != 0 ]; then
-		echo ERROR "error $ret while downloading Intel firmwares"
-		return $ret
-	fi
-	echo DONE
 
 	# now extract MCEdb contents using sqlite
 	_info_nol "Extracting MCEdb data... "
@@ -1001,23 +999,9 @@ update_fwdb()
 
 	# now parse the most recent linux-firmware amd-ucode README file
 	_info_nol "Fetching latest amd-ucode README from linux-firmware project... "
-	linuxfw_url="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/amd-ucode/README"
+	linuxfw_url="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/amd-ucode/README"
 	linuxfw_tmp=$(mktemp -t smc-linuxfw-XXXXXX)
-	if command -v wget >/dev/null 2>&1; then
+	download_file "$linuxfw_url" "$linuxfw_tmp" || return $?
-		wget -q "$linuxfw_url" -O "$linuxfw_tmp"; ret=$?
-	elif command -v curl >/dev/null 2>&1; then
-		curl -sL "$linuxfw_url" -o "$linuxfw_tmp"; ret=$?
-	elif command -v fetch >/dev/null 2>&1; then
-		fetch -q "$linuxfw_url" -o "$linuxfw_tmp"; ret=$?
-	else
-		echo ERROR "please install one of \`wget\`, \`curl\` of \`fetch\` programs"
-		return 1
-	fi
-	if [ "$ret" != 0 ]; then
-		echo ERROR "error $ret while downloading linux-firmware README"
-		return $ret
-	fi
-	echo DONE
 
 	_info_nol "Parsing the README... "
 	nbfound=0
@@ -1042,6 +1026,10 @@ update_fwdb()
 	unset nbfound
 
 	dbversion="$mcedb_revision+i$_intel_latest_date"
+	linuxfw_hash=$(md5sum "$linuxfw_tmp" 2>/dev/null | cut -c1-4)
+	if [ -n "$linuxfw_hash" ]; then
+		dbversion="$dbversion+$linuxfw_hash"
+	fi
 
 	if [ "$1" != builtin ] && [ -n "$previous_dbversion" ] && [ "$previous_dbversion" = "v$dbversion" ]; then
 		echo "We already have this version locally, no update needed"
@@ -6135,10 +6123,15 @@ fi
 exit 0  # ok
 
 # We're using MCE.db from the excellent platomav's MCExtractor project
-# The builtin version follows, but the user can download an up-to-date copy (to be stored in his $HOME) by using --update-fwdb
+# The builtin version follows, but the user can download an up-to-date copy (to be stored in their $HOME) by using --update-fwdb
 # To update the builtin version itself (by *modifying* this very file), use --update-builtin-fwdb
+#
+# The format below is:
+# X,CPUID_HEX,MICROCODE_VERSION_HEX,YYYYMMDD
+# with X being either I for Intel, or A for AMD
+# When the date is unknown it defaults to 20000101
 
-# %%% MCEDB v271+i20230614
+# %%% MCEDB v271+i20230614+b6bd
 # I,0x00000611,0x00000B27,19961218
 # I,0x00000612,0x000000C6,19961210
 # I,0x00000616,0x000000C6,19961210