Implementing Deep Learning Algorithms on Graphics Processor Units

Deep learning has recently become a subject of vigorous research in academia and is seeing increasing use in industry. It is often considered a major advance in machine learning. However, deep learning is computationally demanding and therefore requires highly optimized software and high performance computing hardware. In this paper we share our experience from implementing core deep learning algorithms on a contemporary general-purpose graphics processor units. We describe in details the design decisions and considerations made during the implementation and show that it significantly outperforms high-level matrix-algebra implementations on a typical deep learning task. Finally, we outline a few use cases and research directions that we carried out with this software.