Reinforcement learning for systems and chip design

In the past decade, computer systems and chips have played a key role in the success of AI. Our vision in Google Brain's ML for Systems team is to use AI  to transform the way systems and chips are designed. Many core  problems in systems and hardware design are decision making tasks with massive state and actions sizes. In this talk, I will go over some of our research on tackling such optimization problems.

First, I will talk about our work on deep reinforcement learning (RL) models that learn to do computational resource allocation. The RL agent learns the implicit tradeoffs between computation and communication of the underlying resources and optimizes the allocation using only the true reward function (e.g., the runtime of the generated allocation). I will then discuss our work on optimizing chip placement with RL. Our approach has the ability to learn from past experience and improve over time. To enable our RL policy to generalize to unseen blocks. Our objective is to minimize PPA (power, performance, and area). In under 6 hours, our method can generate placements that are superhuman or comparable on modern accelerator chips, whereas existing baselines require human experts in the loop and can take several weeks.