OCT 17, 2019 | 2:00 PM TO 3:00 PM

Details

Description

Speaker: David Patterson – Turing Award Winner, Google Distinguished Engineer

Topic: Domain Specific Architectures for Deep Neural Networks: Three Generations of Tensor Processing Units (TPUs)

Abstract: The recent success of deep neural networks (DNN) has inspired a resurgence in domain specific architectures (DSAs) to run them, partially as a result of declaration of microprocessor performance improvement due to the ending in Moore’s Law.
DNN have two phases: training, which constructs accurate models, and inference, which serves those models.  Google’s firs generations Tensor Processing Unit (TPUv1) offered 50x improvement in performance per watt over conventional architecture for inference.  We naturally asked whether a successor could do the same training.
This talk reviews TPUv1 and explores how Google built the first production DSA supercomputer for the much harder problem of training, which was deployed in 2017.
Google’s TPUv2/TPUv3 supercomputers with up to 1024 chips train production DNNs at close to perfect linear speedup with 10X-40X higher floating point operations per Watt than general-purpose supercomputers running the high-performance computing benchmark Linpack.
BIO: David Patterson is a Berkeley CS Professor emeritus, a Google distinguished engineer, and the RISC-V Foundation Vice-Chair.  He received his BA, MS, and Ph.D., degrees from UCLA. His Reduced Instruction Set Computer (RISC), Redundant Array of Inexpensive Disks (RAID), and Network of Workstation projects helped lead to multibillion-dollar industries.  This work led to 40 awards for research, teaching, and service plus many papers and seven books.  The best known books is Computer Architecture: A Quantitative Approach and the newest is The RISC-V Reader: An Open Architecture Atlas.  In 2018 he and John Hennessy shared the ACM A.M. Turing Award.

This entry is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.