Since our last update, we have been busy designing the next iteration of our LS1043 platform, called the ‘LS1043S.’ This is a product designed for some of our OEM customers, and while it has some similarities to the Five64, it is designed for embedding inside network appliances and is not as hacker-friendly or featureful as the Five64 will be.

Testing

Our PCB designer, Leon, has been working hard on the LS1043S PCB, prototypes of which are being made as you read this. In the lead up to the PCB being created, we (Guy and Matt) have been testing our LS1043V boards to find any issues that could cause problems in production, including tests for thermal stress, electromagnetic compatibility (EMC), bus signal integrity, and throughput. Read on for more details.

EMC Pre-testing

As the LS1043S and Five64 are intended for end-user use, finding potential EMC issues is important. This includes tests in our lab and a full scan at an accredited test lab, for both conducted emissions (those that escape the system through the attached cables) and radiated emissions. The results of our testing have been used to optimize the LS104S PCB layout.

USB 3 and PCIe Testing

We have tested throughput with USB 3 and NVMe (PCIe) SSD drives. Using a good drive, such as a Samsung 960 EVO, we can come close to the theoretical bandwidth of a PCIe lane:

dd read test, bs=4k: 6628112384 bytes (6.6 GB) copied, 13.8751 s, 478 MB/s (3.824 Gb/s)
fio sequential write test:  WRITE: io=5120.0MB, aggrb=391259KB/s, minb=391259KB/s, maxb=391259KB/s, mint=13400msec, maxt=13400msec
(~391MB/s, 3.13Gb/s)

NVMe Connected to LS1043V via adapter

Note that while NVMe SSDs typically use two or four PCI Express lanes, only one (PCIe 2.0, ~ 500 MB/s) lane is connected for this test.

10G (SFP+)

While our previous LS1043V board did not have an SFP cage, we did route the 10G lane to a backplane connector. We built a SFP+ test board that fits under our current LS1043V board, which validates that our SFP+ frontend is working, as well as provides a platform to develop the related software changes for SFP/SFP+ management. The board pictured below has had some surgery for EMC mitigations - hence some visible modifications!

SFP+ test board

We ran some benchmarks with a 10G uplink feeding five downstream 1G clients (each client running iperf3 -R -c <server>). The results so far are promising:

The results differ depending on the kernel — 4.15 has better performance in general, but maxes out at a lower point for IPv4 NAT vs NXP’s 4.9 tree.

With a bit of tuning it should be possible to improve performance further, and toolkits such as DPDK can push the hardware even further for specific applications.

Next Steps

Overall, we’re pleased with the reliability of our LS1043V boards. The Five64 backers will certainly benefit from the amount of engineering work that has taken place so far. When our next set of prototype boards arrive, we will post an update. While our next prototypes are being made, we’re working on:

DDR Testing

With higher capacity DDR4 chips (4 GB and 8 GB) scheduled for our products, we will be re-working some of our prototype boards with the new RAM chips so we can test and qualify them. This includes stability tests as well as determining the correct initialization parameters, similar to those stored in the SPD EEPROM’s on a DIMM.

BOM Scrub for Production

A shortage of capacitors and other electronic components has caused some components to be quoted at crazy leadtimes, up to 40 weeks! We scrubbed our BOM to ensure our boards can be produced with acceptable leadtimes and with as little impact on pricing as possible.

Meltdown and Spectre

The Cortex-A53 cores inside the LS1043 SoC are not affected by the Meltdown and Spectre issues, as the Cortex-A53 uses in-order execution, a mode that is common among power efficient embedded processors.

The Raspberry Pi blog and ARM Processor security update provide some excellent details on why in-order ARM cores are not effected by these issues.

That said, since other ARM64 cores (e.g., the high-performance Cortex-A72 core) are affected by Spectre and Meltdown, there could be residiual effects as mitigations are added to the kernel. Once these mitigations start shipping we will take a look at the performance impact.

Exploring the Potential for ARM Virtualization

You may have heard about recent progress with ARM in the datacenter market, such as Qualcomm’s Centriq processor and Cavium’s ThunderX. There are already public cloud deployments, such as packet.net and Scaleway.

Both these server-class processors and the LS1043 SoC support the 64-bit ARM instruction set (ARMv8) and its virtualization extensions, meaning the Five64 can run the same operating system images as its datacenter counterparts.

With I/O capabilities that match that of typical x86 machines, including DDR4, PCI Express, and multiple native network ports, we believe the Five64 is well positioned to bring the benefits of the ARM ecosystem to the "edge."

We put together a small, OpenWrt-based distribution called μVirt that runs ARM64 cloud server images that are available today. Potential uses in the home include virtualizing routers, media, and home automation services in the same box. For business, potential uses include "Universal CPE" deployments.

Below is a demostration video. We would like to know your thoughts.