This is the second part of our series detailing the functions and responsibilities handled by the FPGAs on a Talos™ mainboard. In this update, we delve into the unique FlexVer™ trusted computing system used on the Talos™ mainboards. This update builds on information presented in a previous update, "A Word on Lockdown", and we strongly suggest reading both that update and part 1 of this update before proceeding.

Overview

As we mentioned earlier, one of the major reasons for locking down the Core Root of Trust Measurement (CRTM) with vendor signing keys is that the CRTM is a critical part of the trusted boot chain, one that, if compromised, renders trusted computing on the compromised platform undetectably broken. Unfortunately, using the "sledgehammer" approach of locking the CRTM with a vendor signing key also takes effective ownership of the machine away from its physical owner, and, even worse, makes the vendor signing key and/or CRTM code a single, common point of failure across millions of systems at any given time.

Several attempts at solving this problem have been made over time, with the most common being a vendor signed and controlled CRTM and low level firmware stack that allow the operating system keys to be modified by the machine owner. As the CRTM and firmware itself have been growing in complexity, effectivly becoming a hypervisor or operating system in their own right, this is no longer an acceptable situation as the machine owner does not have the ability to modify the CRTM or firmware stack, only to select what operating system signatures are authorised for use. Furthermore, because the vendor continues to control the firmware stack, the ability to select a custom OS signing key at no charge may or may not continue to be present in future updates to the firmware stack; the PlayStation 3 is a perfect example of permanent removal of advertised features after product purchase via a firmware update.

Another notable attempt at solving this problem can be seen in some of the Freescale (then NXP, now Qualcomm) ARM SoCs. These devices allow an owner signing key to be permanently burned into fuses inside the SoC, theoretically preventing execution of any unauthorised firmware. Unfortunately, this also means the CRTM is either hardwired into silicon via, for example, an internal ROM that cannot be audited or inspected for integrity, or, worse, may be updateable via a vendor signed firmware file. Finally, the end goal of preventing unauthorized code execution is not achieved if physical access is available — swapping components (e.g., firmware ROMs or even the SoC itself) between the locked and an otherwise identical unlocked system could bypass the signing checks entirely. This becomes an even greater threat when extended out of the SoC realm into standard socketed processors with an external TPM; bypassing any security checks programmed into the CPU package becomes a trivial matter of malicious firmware, an identical CPU with the malicious keys preinstalled, and a few minutes or less of physical access to the powered down system.

Some systems don’t even attempt to solve this problem or provide any protection against the so-called "Evil Maid" firmware-level attack, and simply state that physical access is equivalent to a compromised device. All owner-controlled systems using coreboot currently fall into this category, as do the existing OpenPOWER machines and the ChromeBook / ChromeBox devices. With the rise in both mobile computing and colocation where machines are not used in environments that are physically controlled by their owners at all times, but are still expected to handle sensitive material, this stance is completely unacceptable. Even where some work has been done, only partial solutions have been created that simply shift the technical risks elsewhere while simultaneously taking away owner freedom. The industry has ignored the problem long enough, it’s time for change!

FlexVer

Background

We chose to tackle the trustworthy CRTM issue directly at the source. At its core, a trustworthy CRTM is simply a chunk of verification code and/or hardware that the machine owner has decided to install and trust as the root of the system security. When stated this way, several key characteristics become obvious:

• The CRTM should be auditable -- no mask ROMs or hardwired silicon should be required
• The CRTM should be tamper evident -- modifications to the CRTM by a third party should be obvious and prevent TPM unsealing
• The CRTM should be integral -- removal or bypassing of the CRTM should prevent TPM unsealing
• The CRTM should be owner controlled -- no vendor signing keys or closed source blobs should be needed to modify the CRTM
• The CRTM connections to the CPU should be physically secure -- a third party should not be able to bypass the CRTM checks with physical access
• The CRTM connections to the TPM should be physically secure -- a third party should not be able to monitor and/or hijack the TPM signals with physical access

Design

System architecture-with FlexVer and LPC guard

Our Flexible Verification (FlexVer™) system addresses all of these points in a novel way. We use an extremely low power FPGA as the core of the FlexVer™ system; when power is applied and immediately after the FPGA soft logic is configured, a cryptograpically random internal private key and associated public key are generated. This key is stored within the FPGA soft logic, and as a result it is irretrievably lost if power is removed from the FPGA. Due to the FPGA’s extremely low idle power consumption, this key can be maintained for years using no power except for the on-board CMOS or NVRAM battery. When the system is plugged into a power source, even if the system is off, the system’s standby power rail is used instead of the battery to limit battery drain and allow for battery replacement. The key is also erasable on detection of physical tampering with the FlexVer™ system, for instance if the physical shield protecting the FPGA and TPM is compromised. Effectively, this key handles the tamper evidence requirement listed above. While FlexVer™ itself is owner controlled and can be modified at will, the cost of this modification is loss of the existing private key. If the machine owner is altering FlexVer™, loss of the key is no problem — the owner can simply reprovision the TPM or other integrity checking systems to accept the newly created key. If an unauthorised party alters FlexVer™, the TPM will remain permanently sealed pending reprovisioning, the machine will not be able to launch its trusted environment, and the machine owner will become aware that tampering has taken place.

As alluded to above, FlexVer™ contains a TPM, and this forms an integral part of the trusted environment. This TPM can be either inside the FPGA or, more commonly, a dedicated external TPM to reduce battery use when standby power is unavailable. The communication lines between the TPM and the FPGA are sensitive, and as a result they are physically protected by placement on the inner layers of the FlexVer™ PCB, and / or by tamper detection using a shield covering the sensitive lines. If tampering is detected, the private key inside the FPGA is immediately erased. Furthermore, the lines between the LPC master and the FlexVer™ system are sensitive, but only from a hijacking perspective as no sensitive key or key-like material is transmitted over these lines. Placement of these lines on the inner layers of the system board is generally sufficient to prevent hijacking.

Finally, FlexVer™ connects to the main system firmware Flash storage device, and additionally may connect to other firmware components such as the BMC or LPC Guard™ firmware devices for additional verification. If a component is protected by FlexVer™, the FlexVer™ system is inserted between that component’s firmware storage and the protected component. This prevents timing attacks, as FlexVer™ will load the critical root section(s) into internal storage for verification, then only allow the critical sections to be read by the protected component from the verified internal memory. All other firmware operations outside of the root section(s) are passed directly through to the relevant firmware storage device(s). As a result of this requirement, FlexVer™ also contains a small RAM; this component may be contained inside the FPGA or, more commonly, provided as an external device to reduce battery use when standby power is unavailable. As before, the communication lines between the FPGA and the RAM are sensitive, and are physically protected by placement on the inner layers of the FlexVer™ PCB, and / or by tamper detection using a shield covering the sensitive lines.

Operation

While a key characteristic of FlexVer™ is its flexibility, and its ability to be modified to handle almost any level of requisite security, we detail the preferred mode of operation here. On detecting a system start request, FlexVer™ begins reading the root section of the main system firmware into its internal memory, while stalling the CPU’s requests for that root section and blocking all TPM traffic. After read is complete, a cryptographic hash is generated from the root section, and this hash is signed by the internal private key. The TPM is initialised, and this signature, or part of it, is then used as the SRTM PCR0 measurement. After PCR0 programming, any TPM startup requests continue to be rejected, while all other TPM traffic is unlocked for direct passthrough. Once TPM setup and PCRO programming is complete, the CPU interface stall is released, and subsequent requests are handled via either access to the internal memory for the root section, or direct passthrough to the firmware storage device for requests outside of the root section. For added security, FlexVer™ by default blocks writes to the root section of the system firmware Flash device, however a dedicated LPC instruction will release this lock. Writes do not take effect from the CPU’s perspective until the next system reboot; dedicated LPC instructions can be used to verify Flash contents prior to reboot. Finally, the firmware hash and signature are stored for later read over LPC if needed.

Integration with other Talos™ Components

In our prior update, we mentioned that FlexVer™ was critical to finally securing LPC Guard™ and the TPM from unauthorised physical access. As implemented on Talos™, FlexVer™ monitors the LPC Guard™ and BMC firmware, and provides the neccessary tamper evidence to prevent hijacking of the LPC Guard™ or BMC systems. When combined, and especially when used with system level protections such as TRESOR, these two technologies make privilege escalation with physical access extremely difficult, while still retaining owner control of the aformentioned machines.

Conclusion

From exclusively using open-toolchain FPGAs to providing auditable security components and a reconfigurable GPIO interface, Talos™ is likely the most secure, auditable, and extensible general purpose workstation / server system ever marketed. Talos™ respects your freedom and privacy, not only through open firmware components and owner-controlled trusted computing, but also through the ability to modify the operation of Talos™ and its external interfaces at an unprecendented level. If you want to help reverse the trend of vendor-controlled and locked machines, while increasing security to a level not seen on any existing owner-controlled systems, please purchase a Talos™ mainboard, system, or POWER8 SSH access today!