The foundation of any embedded design, bootloaders can be a real pain in the butt. So FierceElectronics asked Michael Anderson, an accomplished principal engineer and expert on secure bootloader design who has run workshops for countless embedded developers, to share his insights on what makes bootloader design hard. And how not to let it kill you in the process.
FE: Where in the hierarchy of embedded challenges is bootloader design, or more specifically secure bootloader design?
Anderson: The bootloader is the foundation of any embedded design. Without some way to either load your code or operating system, the embedded platform is just epoxy, solder and silicon. In the bootloader world, you need to be able to read schematics, manufacturer's data sheets and speak both the assembly language for the processor architecture and C—no VMs, no Java, Python, C#, Golang, etc. to be found at this level.
When you add the requirement to make the bootloader secure, you've compounded the problem because you now need to be able to think about security domains, attack surfaces, security keys and digital certificates, and much more. Sometimes, there will be a special mode of the processor such as the ARM TrustZone. In other cases, you'll need to use external hardware such as a Trusted Platform Module (TPM). But, in all cases, security needs to be designed in from the power-on jump all the way to a running OS.
FE: Why is it so hard to get a processor started from the initial power-on jump?
Anderson: Because you need to intimately understand the hardware and its initialization requirements. Chip selects, bootloader locations (many CPUs can boot from multiple devices in the address space), disabling the MMU, initializing the instruction cache but disabling the data cache, disabling the interrupt controller because you don't have an interrupt vector table yet, initializing the front-side bus/north bridge, programming the memory bridge and much more. Most of this is done in assembly language and there aren't many higher education institutions that still teach what assembly language is let alone how to get down to this level of the processor.
FE: Are there some common misperceptions about bootloaders?
Anderson: Probably one of the biggest misconceptions is that RAM exists when you power the processor on. It doesn't exist until you set up the chip selects and initialize the cache controller/memory protection unit/memory management unit(s). Until that time, you don't even have a stack. So, you're stuck with the register and instruction set of the processor and that's it.
I once worked on one SoC that needed 20,000+ instructions just to light the chip up enough to switch to C! You're working at the lowest level of the processor here. And, if you don't get the refresh timing right for dynamic RAM, your memory will just fade away--a little bit at a time. This can significantly complicate debugging.
FE: What additional challenges do multi-core and security add to the mix?
Anderson: It's tough to find a single-core processor these days. So, cache coherency is a major battle if you've got multiple processors running when you start the boot cycle. Therefore, the simplest approach is to put the other processor cores to sleep while one master processor handles all of the low-level code and initialization. In fact, most bootloaders won't even bother with the other cores beyond putting them in reset mode. That's the problem of the OS to light up the other cores.
FE: What three things should every embedded developer know about bootloader design to be better at it?
Anderson: One: Avoid assuming anything about the state of the processor when you power on. Some processors have some microcode that they run automatically on power up, but many do not.
Two: Understand the entire power up sequence for the processor before you get started and know when you plan to transition from assembly to C.
Three: Understand all of the chip's peripherals and their initialization requirements before writing any code.
FE: And what is your advice for the most dreaded circumstance of all--needing to update a bootloader in-the-field?
Anderson: This is something you have to plan for from day one. You'll need to have fail safe conditions to deal with the inevitable case of losing power during a firmware upgrade. Manufacturers will often times blow the e-fuses on the debugger interfaces.
So, you'll need some rock-solid pre-loader to boot strap the firmware back into existence using one of the peripherals like a serial port, USB port or network interface. Make sure you test this process with non-engineers to ensure that you've covered all of the possible mistakes the average consumer could make while trying to recover the device. No manufacturer wants angry customers sending bricked devices back to the factory for firmware recovery.
Michael Anderson and Jacob Beningo will be teaching an all-day workshop “Secure Bootloader Design from MCUs to Application Processors to Keep the Bad Actors Out,“ at the Embedded Technology Conference @ Sensors Expo & Conference June 22-24, 2020 in San Jose, CA.
For more information visit the Embedded Technologies Conference website.
As a FierceElectronics reader, you are eligible to use the following promotion code to receive $100 off a conference pass or for free entry to the Sensors Expo Hall: FE100
Note that Early Bird Rates for the Embedded Technology Conference @ the Sensors Expo and Conference are in effect until April 17, 2020. Passes are limited – so act fast! Register here.