Fengshui: Demystifying Chiplet Ecosystem and Bespoke Neural Network Accelerator Codesign

Fengshui is a chiplet ecosystem and accelerator co-design framework that jointly optimizes chiplet pool composition and bespoke application-specific integrated circuit (BASIC) design. The paper argues that operator-level disaggregation—giving each operator its preferred microarchitecture, batching strategy, and memory hierarchy—is essential for meeting the latency and energy constraints of modern ML workloads, but the resulting bespoke accelerators incur prohibitive NRE costs. Chiplet-based integration can amortize these costs, but choosing which chiplets to build and how to compose them are circularly dependent problems. Fengshui addresses this by co-exploring chiplet and memory heterogeneity, tensor fusion, and pipeline/tensor/expert parallelism, with place-and-route validation for physical implementability. With just 8 strategically selected chiplets—including network switches, processing-in-memory units, and accelerators with diverse microarchitectures—Fengshui-generated BASICs achieve 53.8%, 57.6%, 84.3%, and 90.6% reductions in energy, energy×cost, EDP, and EDP×cost over homogeneous accelerators, while scoring within 2% of unconstrained heterogeneous designs. Case studies show 15–19% energy and 35–39% energy×cost reductions for datacenter LLM serving, and 25.5% energy×cost and 10.5% energy reductions for edge autonomous vehicle perception while meeting real-time deadlines.