Project update 20 of 28
Manufacturing & Delivery
Dear Backers,
First, a sincere apology for missing our target of shipping Novena mainboards in November. bunnie in particular expresses his regrets for slipping on such a major milestone.
However, the good news is we are just a couple of weeks delayed. The material for the mainboards is all in the factory, and first articles have been delivered off of the production line and approved. This week, all of the material went through the SMT process, and next week our factory will be doing manual insertion and testing.
We’re looking at the second week of December for material to be forwarded from our factory to Crowd Supply in Portland, Oregon. From there, the material will be mailed to sites around the world. International customers should expect up to an extra month for delivery due to holiday and customs for the packages.
Other boards are also in the process of assembly, including the Senoko battery board and the GPBB.
We also thought it would be interesting to share this photo from our factory, which is a water cleaning station for the PCBs after SMT. Many people have an impression that water will ruin electronics, but in fact pure, clean water is no problem for most electronic parts that are process-sealed. Electronics are destroyed by exposure to water typically due to a combination of the salts in the water, ionic contaminants deposited on the PCB during the course of normal use, and in the case of battery-powered devices, the electronics being powered on during exposure.
Another interesting tidbit from the factory floor is the Novena test procedure. Every board goes through a battery of almost two dozen tests, the source code for which is shared on xobs’ Github account. We try to exercise every aspect of the board, with a focus on components we deemed to be particularly tricky to assemble or prone to other yield issues. In addition to exercising every port and all the internal components from the RTC to the FPGA, the final test program burns an initial firmware image onto Novena.
xobs labored tirelessly to create an automated framework for packaging up a base Debian system, also shared on Github, as well as creating a first-boot experience that accommodates multiple user scenarios, from serial consoles to HDMI monitors to a built-in LCD panel. Built into the initial firmware is Jon Nettleton’s initial release of a package that enables accelerated desktop window management via a DDX driver.
We also wanted draw attention to one cool project brewing in the Novena user community. The sourcerer has created an Altium-toKiCAD converter, which you can read more about here. It works well for converting schematics, and the PCB conversion feature is starting to look good, too. We’re excited to see the emergence of a tool that so greatly improves the accessibility of Novena’s hardware design source. There’s still more work to be done on the tool, so if this is a topic of concern to you, we encourage you to check it out and contribute to the effort!
We think we’re still on track for delivering our first round of desktop units in late December. Most of the material is staged in the factory and we will begin first article assembly right after the mainboard assembly process is completed. Production for the desktop units runs a bit slower due to the large amount of manual labor required to put the devices together, so we will continue to produce and deliver desktop units into January at which point we will retool to produce the laptop units.
We still haven’t answered the question of why we’re late. It’s hard to pin down any single event that torpedoed the schedule – Novena is a very complex assembly with hundreds of parts and dozens of processes. If any component is delayed or missing, the whole process is delayed; it’s frustrating and humbling how hundreds of thousands of dollars of inventory can be held up on the account of a single missing component the size of sand grain. To give you an idea of how complex Novena is as an assembly, here is the top-level supply chain map for just the laptop assembly; keep in mind that beneath this map are PCB BOMs with sometimes hundreds of line items.
About half a dozen factories are involved in the fabrication of custom components for Novena scattered all around the globe. Contrary to popular belief, we don’t work exclusively with factories in the Pearl River Delta of China – we build supply chains that make the most sense for the scenario. For a lot of reasons we’re using a hybrid supply chain for Novena. We build the mechanical subassemblies in China, run SMT in the US, and perform final assembly, test and pack-out in the US.
The net result is that while it’s easy to blame our delay upon a holiday season slowdown, that’s kind of a cop-out. Thanksgiving wasn’t a surprise, and in fact our plans had originally factored that in. We hit a lot of road bumps along the way, from delays in wifi card sourcing, to last-minute bug fixes, to agonizing decisions over quality improvement vs. schedule, to customs problems in China, to getting a team of engineers up to speed on the sheer complexity of Novena, to basic communication failures across time zones, cultures, and languages. On the other hand, we had many victories on the way, from our extremely diligent and hard working team of engineers who would trade emails till the wee hours in the morning, to all of our PCBs having first-run power on success, to our sourcing team pulling miracles at the 11th hour to marshal material across the globe into assembly facilities just in time. It just works out that our failures outnumbered our victories in this campaign, and while perhaps we could have cracked the whip a bit harder and pulled in the schedule to meet the deadline, it would have been a Pyrrhic victory – at the end of the day, supply chains are made of people, and we believe these people should not be treated like slaves. We also acknowledge the fallibility of people, but also believe that growth comes of learning from mistakes and lasting partnerships are forged from shared burdens.
At any rate, we continue to make progress, and we’re doing our best to try to meet all of our remaining milestones on-time. Thank you for your continued support and patience. If you have questions or comments, please remember we do have a forum and we’re on #kosagi on OFTC. We’ll also be at 31C3 this year, sitting at the fail0verflow table, if you happen to be in attendance.
-bunnie & xobs