Suntec Singapore Convention and Exhibition Centre
Tutorial Pass - $250
Fundamentals of Deep Learning for Multi-GPUs
Presented by SGInnovate, NVIDIA & NSCC
SGInnovate, together with the NVIDIA Deep Learning Institute (DLI) and National Supercomputing Centre (NSCC) is proud to bring to you Fundamentals of Deep Learning for Multi-GPUs.
Led by seasoned instructors, you can expect to learn the latest deep learning techniques for designing and deploying neural network-powered machine learning across a variety of application domains.
This workshop teaches you to apply techniques to train deep neural networks on multiple GPUs to shorten the training time required for data-intensive applications. You will work with widely-used deep learning tools, frameworks, and workflows by performing neural network training on a fully-configured GPU accelerated workstation in the cloud. The workshop starts with a linear neuron, defining the loss function and optimisation logic for gradient descent. It then teaches the concepts to transform single GPU to Horovod multi-GPU implementation to reduce the complexity of writing efficient distributed software and then concludes by teaching the techniques to improve the overall performance of the whole pipeline.
In the course, participants will learn:
- Various approaches to multi GPU training
- Algorithmic and engineering challenges to the large-scale training of a neural network
08:45am – 09:00am: Registration
09:00am – 09:45am: Theory of Data Parallelism
Understand the issues with sequential single thread data processing and speeding up the applications with parallel processing.
- Issues with sequential processing
09:45am – 10:00am: Tea Break
10:00am – 12:00pm: Introduction to Multi GPU Training
Define a simple neural network, and a cost function and iteratively calculate the gradient of the cost function and model parameters using the Stochastic Gradient Descent (SGD) optimisation algorithm.
- Overview of loss function, gradient descent, and SGDs
12:00pm – 01:00pm: Lunch
01:00pm – 03:00pm: Algorithmic challenges to Multi GPU training
Learn to transform single GPU to Horovod multi-GPU implementation to reduce the complexity of writing efficient distributed software. Understand the data loading, augmentation, and training logic using AlexNet model.
- Data parallelism
- Large minibatch and its impact on accuracy
- Gradient exchange
03:00pm – 03:15pm: Tea Break
03:15pm – 05:15pm: Engineering Challenges to Multi GPU Training
Understand the aspects of data input pipeline, communication, reference architecture and take a deeper dive into the concepts of job scheduling.
- Keeping up with the GPU
- Job Scheduling
- Overview of the wider AI system design
05:15pm – 05:30pm: Closing Comments and Questions
A quick overview of the next steps you could leverage to build and deploy your own applications and any Q&A
- Wrap-up with the potential next steps and Q&A
- Workshop Survey
This workshop is organised in conjunction with SCAsia 2019.
Kindly register under “Tutorials” in Registration Type.
After clicking “Attending” under “Tutorial Pass at SGD$250/pass (11 Mar)”, please select “Fundamentals – Multi GPU”.
You may also like the following:
Fundamentals of Deep Learning for Natural Language Processing
SGInnovate, together with the NVIDIA Deep Learning Institute (DLI) is proud to bring to you Fundamentals of Deep Learning for Natural Language Processing.Topics:
AI Roundtable: Corporate-Startup Collaboration
Artificial Intelligence (AI) is on the rise, and very rapidly so. This technology for next-generation intelligence and decision making is permeating every industry and much of human activity. AI technology is also getting more advanced increasingly fast, creating more and more possibilities and opportunities.Topics:
Canada’s Perspectives on Advanced Vehicle and Industry Technology
Global industries and governments are racing to bring self-driving vehicles to their streets and – just as importantly – competing to make the greatest strides to capture that market. The 2018 KPMG Autonomous Vehicles Readiness Index (AVRI), which rates major industrial countries on their ability to adopt self-driving vehicles en masse, ranked Canada seventh in overall readiness for a world of self-driving vehicles. Canada’s advantages for developing autonomous technology include a highly skilled workforce and leading work in artificial intelligence.Topics:
The Future of Finance: How FinTech Initiatives Are Driving Innovation in Financial Services
The financial sector has been making bold bets on the digital revolution. For this event, we bring together the brightest and best fintech professionals from universities and the industry for an afternoon of thoughtful discussions, knowledge sharing, and networking. This session will cover both technical updates and market trends. Come join us and stay ahead of the competition!Topics:
Holland Innovation Seminar: Exploring collaboration opportunities in the field of Artificial Intelligence
Singapore and the Netherlands are both ranked in the top five of the most innovative economies in the world on the Global Innovation Index. Both countries attach great importance to innovation and the triple helix approach, bringing government, academics and industry together to collaborate. How can we deepen and expand our R&D efforts, especially in the field of AI?Topics:
Artificial intelligence is already a part of our everyday lives, but for it to truly make a difference it needs to understand human emotions, says Live with AI founder Pierre Robinet.
This article is written by Pierre Robinet, who is a senior consulting partner at Ogilvy Consulting and founder of Live With AI, an independent think tank based in Singapore.