Training Large-Scale Deep Neural Networks

Dr. Liang Zhao, an assistant professor in the Department of Computer Science, has focused research on datamining, artificial intelligence, and machine learning for the past several years. His most recent project is centered on training of large-scale deep neural networks (DNNs) and its applications to graph-structured data such as large-scale social networks and commercial networks. Due to the complexity and scale of this endeavor, Dr. Zhao chose to partner with AWS at Emory to gain the tools needed for his work.

In recent years the constant improvement of DNNs’ performance for graph-structured data is accompanied with the fast increase of the models’ complexity and size in order to handle applications with ever-larger real-world networks. This includes recommendations in social networks, biological neural network classifications, and transportation network predictions. Moreover, emerging techniques in the field extended from image deep learning have been tailored to graphs such as graph pooling and graph U-nets, which allow us to further deepen graph neural networks. All the above trends lead to a severe challenge for large models to be fit into a single computing unit (e.g., GPU), and creates an urgent demand for partitioning the model into different computing devices and parallelizing the training. 

The flexibility and scalability of AWS at Emory provided Dr. Zhao with the resources needed to continue his work. "I really enjoy the experience working with the AWS Team, and I am also highly impressed by the AWS at Emory Service Team."

Flexible scheduling and dynamic work re-partitioning are cornerstones to his revolutionary approach. Emory’s unique platform ensures that Dr. Zhao has the right resources at the right time to continue trailblazing in this dynamic and evolving field.