Category: Hardware and compute

This category contains 5 pages.

• AI and Compute · OpenAI's May 2018 analysis by Dario Amodei and Danny Hernandez showing training compute doubling every 3.4 months since 2012, the trendline that founded the scaling discourse.
• Cerebras Systems · American wafer-scale AI hardware company behind the WSE chips, Cerebras-GPT, the Condor Galaxy supercomputers with G42, and a high-speed inference service.
• Distributed training · Techniques for training models across thousands of accelerators: data, tensor, pipeline, and expert parallelism, plus sharding schemes like ZeRO/FSDP.
• Huawei · China's sanctioned hardware champion: Ascend AI chips, the CloudMatrix rack systems, and the PanGu model line.
• NVIDIA · The dominant supplier of AI training and inference hardware; its datacenter GPUs are the substrate of the LLM era.