There are two laws, actually observations, that drove processor design and implementation in the last decades: Dennard scaling and Moore's law. Dennard scaling states that “The power density for a given silicon area remains constant when you increase the number of transistors." and ended around 2003/2004. Consequently, performance growth due to increasing clock speeds was no longer economical, and for large chips it is even not possible to switch on all transistors (“dark silicon” effect) due to the limited energy budget. Processor architectures moved away from more aggressive exploitation of instruction-level parallelism to other, more explicit forms of parallelism, such as data and thread-level parallelism, to turn the increasing number of transistors into more performance. However, more recently Moore's law, known as “The number of transistors per chip doubles every (12) 18 months.” is slowing down as well, resulting in a slower increase in the number of available transistors.
To still be able to increase performance and energy-efficiency, there is currently a strong interest in domain-specific architectures (DSA). These are programmable hardware architectures tailored to a specific application domain.
An important domain with a lot of commercial interst and pressing computation demands is deep neural networks (DNN). In this seminar, we will look at the main concepts for DSAs and then study a number of DSAs for DNNs, analyze which architectural techniques they employ and what level of programming support is available.