Ten64 is well-equipped for continuous integration and continuous delivery. Here’s a detailed look at the full hardware and software setup we use for CI/CD on Ten64. But first, some details on an Ask Me Anything (AMA) this Wednesday and recently added NVMe SSD options that can ship with your Ten64 order.
Ask Us Anything on October 7
We will be holding an AMA session on the Crowd Supply Discord server on Wednesday, October 7, 2 PM US Pacific Time. For those in other timezones, this is:
- 4 PM US Central
- 21:00 UTC October 7
- 11:00 PM Central European Summer Time
- 8:00 AM October 8 Australian Eastern Daylight Time
- 10:00 AM October 8 New Zealand Daylight Time
Or, you can determine the time in other timezones.
The session will be on the Crowd Supply Discord server, channel #ten64-ama. Join or invite other to join with this link: https://discord.gg/ZKjZcwP
After the AMA timeslot, the discussion will be archived on the Crowd Supply Discord.
Ten64 project architect Mathew McBride will be on hand to answer all your questions!
NVMe SSDs Now Available
We’ve just made available NVMe SSDs that are known to work with Ten64. Like other accessories, these drives will only ship with Ten64 orders - they won’t ship separately due to their discounted price. If you’ve already place an order and want to add a drive to your order, please contact Crowd Supply to request an order change.
To be clear, one of these drives will fit within the standard Ten64 enclosure. If you are looking for drives suitable for a NAS in a different enclosure, stay tuned - we’re working on sourcing those as well and will announce them once they’re available.
Continuous Integration & Delivery
If you have done any operating system and/or distribution development, you know that the compile→install→reboot cycle can be quite a chore. Virtual machines can help avoid having to use physical hardware, but sometimes you need to test features that are closely tied to the hardware. Another common scenario is that you are working from home and need to control a system in your office lab.
Ten64’s control header helps address both these scenarios and more by bringing out the reset and power button lines, UARTs, I²C, and other control nets. You can wire these up to your favorite single-board computer (SBC) for remote control and access.
Apart from continuous integration and continuous delivery (CI/CD) you can also use the control header for BMC/IPMI-like applications. Regrettably, we didn’t have enough room on the Ten64 to fit an onboard BMC, but this is the next best thing!
Raspberry Pi Control HAT
We have designed, prototyped, and tested a small Raspberry Pi-compatible HAT board allowing up to two Ten64 systems to be controlled from one Raspberry Pi or other compatible SBC. Though we don’t have currently have plans to manufacture and sell these HATs, we’ve made available the complete Altium project files for the Ten64 Control HAT for anyone who’s interested. If you’d like to see these go into production, let us know!
The control HAT also isolates the I²C buses and GPIOs (using MOSFETs and I²C muxes) between the two systems so things like reboots resetting GPIO states and unconfigured pinmuxes will not disturb the "target" board.
Up until version 4, Raspberry Pi only had a single UART available. To get around this, you can use Ten64’s USB-C console. We use the Seeed Studio NPi i.MX6ULL Dev Board SBC, which provides multiple independent UARTs.
CI/CD Software Tools
On the software side, the tools you need are:
mTerm: a terminal multiplexer which allows multiple clients to connect to the same UART. This utility originates from Facebook's openBMC distribution.- You will need to configure a telnet server to route connections to
mTerm, or perhaps configure an SSH user. - A "stub" Redfish server which will accept reboot and other requests over HTTP. I haven't implemented any other Redfish functionality (yet).
You can get OpenWrt versions of these packages from my openwrt-bmc-feed repo, though the same utilities will also work on other Linux distributions.
Beware that anyone who can access your "host" board (SBC) will have full control of the "target" board (Ten64) as well. Ensure you have secured your network and systems appropriately – these are not intended to be accessed from the open internet.
For Ten64s dedicated to CI/CD testing, I recommend configuring U-Boot to halt at the boot prompt so the testing script can decide the correct course of action. For example, first reboot into the recovery firmware to download and write your image, then execute a normal boot sequence on reboot.
Testing Distribution Images with GitLab CI
Our distribution-images repository builds and tests our customized Debian images against real hardware. Our GitLab CI configuration and testing utility scripts are both available to use as starting points for your own setup.
In our CI/CD pipeline, since a virtual machine can be provisioned and booted faster than real hardware, we first use μVirt to test the images as a VM simply to ensure the image is bootable and basic features like disk image resizes and cloud-init work as intended.
These scripts require the GitLab runner to have access to the control SBC. The easiest way to do this is to have a GitLab runner instance on the same network segment as your control SBC.
