Prediction based Execution on Deep Neural Networks
Authors: Mingcong Song, Jiechen Zhao, Yang Hu, Jiaqi Zhang, and Tao Li Venue: ISCA 2018 The authors of this paper present a technique to further reduce computations within deep-neural network computation, and present a scale-out design which is able to achieve 2.5X speedup compared to traditional accelerators, and 1.9-2.0X compared to Cnvlutin/Stripes. The concept is based on removing the computations of ineffectual neurons (iEONs). In order to do so, they use first compute the upper bits to determine if the result is likely to have a non-negative value. This is exceedingly elegant, as this means the predictor for computation is actually part of the computation itself--no extra work is being performed. Not only that, the technique requires no retraining and incurs no accuracy loss. The caveat is that the number of upper bits for prediction must be experimentally determined (still no retraining though). The challenge is that the iEONs are typically randomly dispersed throug...