Superhuman Hardware Design Is The Easy Part

We’re excited to announce that Orchestra Bio has been acquired by OpenAI. To read more about the acquisition, read Jay Rughani and Jorge Conde’s post about it here. For more general information on what OpenAI is up to in the Health space, check out our podcast with them here. It took me eight months to build my first FPV drone. I procured the parts in January, received them in February, remembered I bought them in June, assembled and lit the drone on fire in July, re-assembled and crashed it into my neighbor’s roof in August, and then, and only then, at the behest of my wife, downloaded a FPV flight simulator.A super-intelligent AI surely could have done better.While far from superhuman today, the frontier models can now generate CAD, design custom PCBs, source COTS parts, and describe how those parts are to be assembled. And they’re only getting better. Hardware design benchmarks like CAD Arena, EE Bench, and my own LuxoBench show steady improvements; and more generally, you’d have to be crazy to bet against the Bitter Lesson and say hardware design will forever stay beyond the frontier.Superhuman hardware engineers are coming soon, perhaps very soon. But what will they look like? Will superhuman hardware engineers surpass all humans in one fell swoop, or only in specific domains/disciplines? And what will hardware engineering look like before, during, and after the crossover?My answer to each of these questions starts with one simple observation: that the interface between the digital and physical world is lossy. This makes it hard to “write” changes from the digital world of bits into the physical world of atoms, and hard to “read” measurements of the physical world back into a computer. This lossiness will shape when and how future superhuman hardware engineers will emerge — and means that even once the models can consistently one-shot electromechanical designs, they will still have a lot more work to do.When OpenAI released GPT-6 Astra, the online world erupted with demonstrations of its 3D design prowess. Tom Krcha used Astra to model a steam train, Thomas Ricouard made a walk-through of a photorealistic house, Anshu Chimala made a 3D game, and Sharif Shameem made the Palace of Fine Arts in San Francisco.These demonstrations got hardware engineers wondering if the models could also generate engineering-grade CAD. Nim Cheema, co-founder of Dessn AI, asked Astra to create a DJ controller. Roberto Nickson made the top models design, engineer, and 3D print bridges. And I launched luxobench.com, where I asked OpenAI’s Astra, Anthropic’s Fable 5.1, and SpaceXSI’s Grok 4.6 to help me build an interactive desktop lamp.We quickly found that — unfortunately — the models by themselves are still not quite ready for primetime hardware engineering.There are many levels to building real-world hardware, and clearing each one requires different, and often entirely independent, skills.At the 101 level, you have to decompose the product down into components, identify available commercial-off-the-shelf (“COTS”) parts, ensure their mutual compatibility, design the mechanical and electrical interconnections between them, ensure the final design is buildable, and then actually procure parts from online stores.Even this step is hard enough, and can set back not only backyard hobbyists like yours truly, but also the most sophisticated engineering teams in the world. In Walter Isaacson’s biography of Elon Musk, he relays a story of a time in 2006 when a Falcon 1 hot fire was delayed due to a capacitor — a tiny electrical component that was only in stock at a store roughly 6,000 miles away from their launch pad in the middle of the Pacific Ocean. Getting COTS parts where you need them, when you need them, is hard enough.Omelek Island, site of the first Falcon 1 launchesAt the 201 level, however, things get even harder: you now have to make custom parts, which means navigating the complex translation step from digital design tools to physical machines. Take, for example, CNC machining. After a mechanical engineer designs a bracket, they can’t just hit “print” and have the part materialize before them. Instead, they work with a machinist, who translates their CAD file into CAM (“Computer-Aided Manufacturing”) software, which writes the instructions that will actually drive the CNC machine.A CNC machine at Hadrian’s factoryIn practice, this is an artisanal process that requires a master machinist to choose the right stock, workholding, tools, and cutting strategy to guide the tools as they cut through the block of titanium. This causes variability in the manufacturing process, and variability leads to errors in translation.At the 301 level, you take those simple parts and start to bring them together into a complex electromechanical assembly. As you add more parts together, you add complexity and “stack” tolerances up — meaning, every part might individually meet its drawing, but still fail to fit into the assembly due to compounded and correlated errors. Those other parts are also likely built by different people on other teams in the company, requiring requirements, schedule, and risks to be communicated across the organization to ensure that the parts can interface nicely together and are delivered when they are needed, but not before and not after.At the 401 level, you realize that your design needs not only COTS parts and custom-designed parts, but also either inputs or particularly complex assemblies from other external vendors. The hardware world loves talking about vertical integration, but basically everyone relies on vendors — it almost never makes sense to build your own chip fab or copper mine. Even SpaceX, the champion of vertical integration, has 591 current employees with a “supply chain” or related titles, and is actively hiring 128 more.1 And once you have outside vendors involved in building custom parts, good luck!And then, finally, at the 501 level, you have to do all of the above at massive scale. It is a fundamentally different task to build 1 of something and to build 1,000,000 of something: you need to worry far more about part and process variability, you need much more financing (much further in advance) for things like machines and inventory, you need to train the people (or program the machines) who will assemble, integrate, and test the product for you, and you quickly hit the limits of your vendors.Each of these “writing” problems, from the 101 to 501 level, are all fundamentally solvable but need to be solved individually. As we’ll discuss below, superhuman AI might be great at pure design, meaning at defining the platonically ideal part for a particular job, but struggle with “design for manufacturing,” or finding a point design that meets the requirements while remaining easy to build/procure and inexpensive. The former is a relatively tractable task that can largely stay within a machine; the latter requires significantly more interaction with the real world, and should therefore prove more difficult for the models to master.One of my friends buys 45 million chickens per week. But despite his efforts to buy the same spec from the same suppliers and run everything through the same process with the same machines, his week-to-week variance in saleable food products after processing (i.e., chicken breasts you can buy at the supermarket) has historically been as high as 20%.Improving that number over time has taken him a herculean effort: extensive measurement and analysis of every step of his supply chain, plenty of old-fashioned vendor cajoling, and a very sophisticated data operation behind both. In one memorable case, he found a malicious supplier deep in his supply chain that was injecting its chickens with water before shipping them. All of the intake measurements for weight would pass, but that water was, of course, not meat, so he was left with less usable product than he intended to buy.Everyone who has built hardware at scale, or otherwise managed a supply chain, has their own version of this story. It is exceptionally hard to know precisely what is going on in the real, physical world.First, the real world is non-uniform, and constantly changing.Physical objects have defects, change their properties based on environmental variables (e.g., temperature), and are subject to wear and tear. Even extremely well-tuned processes and measurement tools stray over time. In practice, this means that physical systems like supply chains and manufacturing lines constantly break, and must be designed with variance in mind — and you can only know when something breaks or has strayed if you invest significantly in measuring the inputs and outputs of the system.Second, real-world data is sparse. You can’t get all the data all the time. This is a fact I’m acutely aware of, having worked on a number of satellite programs: our thermal team would have loved to put thermistors more or less everywhere on the satellite, but the software, structures, and production teams unfortunately also got a say.Astranis satellitesAnd then when something goes wrong, you often realize that diagnosing the issue would have been significantly easier if you had collected some obscure piece of data — a picture of an electronics board at an intermediate stage of assembly, an extra sensor at a new location, and so on.Third, real-world data is often expensive to collect. High-performance domains (e.g., F1 racing) and high-reliability domains (e.g., human spaceflight) often need hardware measurements to be extremely precise, and precision comes with cost in dollars and in time because precise measurements need even more precise machines. For instance, the CMMs (coordinate measuring machines) that measure flatness, perpendicularity, dimensions, and so on for machined parts are not cheap, and often painfully slow to procure and to operate.Fourth, the real world has inherent latency. Physical processes take time — you can only thermally cycle