Graphics processor based implementation of massive neural networks with CUDA

Abstract

This report will present a convolutional neural network based model originated from Yann Lecun and its application in hand written digits recognition. In this model, back-propagation algorithm will be employed to train the neural networks, adjusting the weights in each hidden layer to achieve a high recognition accuracy. In order to solve the problem that it requires too much time to obtain well-tuning variables, a highly parallel programming technique with the use of Graphic Process Units (GPUs), named as asynchronous computing, will be employed to implement our convolutional neural network. I have done a speed comparison between one single CPU and GPUs with the model of spinodal decomposition, in conditions of two dimensions and three dimensions respectively, verifying the significant role GPUs can play in speeding up computation when manipulating huge numbers of data and complex algorithms, up to 60 times acceleration according to my result. With our fully parallelized convolutional neural network, it has shown about 15 times speed-up compare to the code in Matlab version when training the neural network with 60,000 training images from MNIST, and achieved about 99% accuracy after 100 epochs based on 10,000 testing images.