Large Scale Artificial Neural Network Training Using MultiGPUs
Authors: Linnan Wang (Georgia Institute of Technology), Wei Wu (University of Tennessee, Knoxville), Alex Zhang (Rutgers University New Brunswick), Jianxiong Xiao (Princeton University), Yi Yang (NEC Laboratories America)
Abstract: This paper describes a method for accelerating large scale Artificial Neural Networks (ANN) training using multi-GPUs by reducing the forward and backward passes to matrix multiplication. We propose an out-of-core multi-GPU matrix multiplication and integrate the algorithm with ANN. The experiments demonstrate that the proposed algorithm can achieve linear speedup with multiple inhomogeneous GPUs.
Two-page extended abstract: pdf