In the early 1980s, when computer-aided engineering (CAE), the precursor to modern electronic design automation (EDA), was just taking shape, my professional trajectory shifted in a way that would prove foundational. I joined Teradyne, the Boston-based leader in automated test equipment (ATE), and I encountered for the first time the world of logic and fault … Read more
For the past decade, the semiconductor industry has been moving in one direction: shift-left, specifically, shifting more validation into the pre-silicon phase. The idea was straightforward: if software ultimately determines how a system behaves, then software should become a primary vehicle for verification. The industry responded by investing heavily in virtual platforms, emulation systems, and … Read more
Any discussion of modern AI system performance must include MLCommons and its MLPerf benchmark suite, which has become the industry’s de facto standard for measuring machine learning performance. Since its debut in 2018, MLPerf has provided a neutral, peer-reviewed framework for comparing hardware and software platforms across a broad range of AI workloads. The original … Read more
Recent frontier LLM inference benchmarks have highlighted a recurring pattern. GPU-based systems deliver outstanding throughput when latency is not a concern, but their performance drops sharply once real-time response requirements are imposed. This behavior is sometimes attributed to software inefficiencies or suboptimal system tuning. In reality, the root cause lies much deeper. It reflects a … Read more
There is a growing architectural tension at the heart of modern AI infrastructure. The processors that enabled the deep learning revolution—graphics processing units (GPUs)—remain the dominant engines of large-scale training and inference. Yet the computational profile of frontier language models is evolving in ways that increasingly expose the structural assumptions embedded in GPU design. Memory … Read more
Hardware emulation arose as a necessity out of the needs of the eighties. By the mid-1980s, semiconductor designs had outgrown the practical limits of gate-level simulation. Gate-level simulation delivered accuracy, but at glacial pace; silicon prototypes performed at real-speed but arrived far too late. The industry needed a new instrument, a verification engine capable of … Read more
FPGA prototyping and hardware emulation originated from two independent demands that emerged at roughly the same time, namely, the necessity to implement digital designs in reconfigurable hardware. This was conceivable given the newly introduced field programmable gate array (FPGA) device. Yet from the very beginning they were driven by different motivations. Hardware emulation emerged from … Read more
The AI industry has converged on a deceptively simple metric: cost per token. It’s easy to understand, easy to compare, and easy to market. Every new system promises to drive it lower. Charts show steady declines, sometimes dramatic ones, reinforcing the impression that AI inference is rapidly becoming cheaper and more efficient. But simplicity, in … Read more
America’s data centers managed to keep their electricity use surprisingly steady from 2005 to 2017, with rather small annual increments contained via constant improvement in electronics. Then, around 2017, AI arrived forcefully and disrupted that stability. AI required a different computing machine, one designed not for ordinary tasks, such as our beloved PC, but for … Read more
A recent analysis highlighted by MIT Technology Review puts the energy cost of generative AI into stark perspective. Generating a simple text response from Llama 3.1-405B—a model with 405 billion parameters, the adjustable “knobs” that enable prediction—requires on average 3,353 joules, nearly 1 watt-hour (Wh). Once cooling and supporting infrastructure are factored in, that figure … Read more