Modeling and Analyzing Evaluation Cost of CUDA Kernels

General-purpose programming on GPUs (GPGPU) is becoming increasingly in vogue as applications such as machine learning and scientific computing demand high throughput in vector-parallel applications. NVIDIA’s CUDA toolkit seeks to make GPGPU programming accessible by allowing programmers to write GPU functions, called kernels, in a small extension of C/C++. However, due to CUDA’s complex execution model, the performance characteristics of CUDA kernels are difficult to predict, especially for novice programmers.