Advanced Natural Language Processing and Temporal Sequence Processing | SGInnovate
July222019
July232019

Location

BASH, Level 3,
79 Ayer Rajah Crescent
Singapore 139955

Price

Early Bird Module 2 [Ends on 30 June 2019] (Ticket Inclusive of G.S.T) - $1524.75
Module 2 (Ticket Inclusive of G.S.T) - $1605

Advanced Natural Language Processing and Temporal Sequence Processing

Organised by SGInnovate and Red Dragon AI

Together with Red Dragon AI, SGInnovate is pleased to present the third module of the Deep Learning Developer Series. In this module we dive deeper into some of the latest techniques for using Deep Learning for text and time series applications.

About the Deep Learning Developer Series:

The Deep Learning Developer Series is a hands-on and cutting-edge series is targeted at developers and data scientists who are looking to build Artificial Intelligence applications for real-world usage. It is an expanded curriculum that breaks away from the regular 8-weeks long full-time course structure and allows for modular customisation according to your own pace and preference. In every module, you will have the opportunity to build your own Deep Learning models as part of your main project. You will also be challenged to use your new skills in an application that relates to your field of work or interest.

Start here

    • Have an interest in Deep Learning?
    • Join us if you are able to read and follow codes
    • This module is compulsory before you take the advanced modules
     
    • You will need to take module 1 before this module
     
    • You will need to take module 1 before this module
     
    • You will need to take module 1 AND module 2 OR 3 before this module
     
    • You will need to take module 1, 2 AND 3 before this module
    • Attain a “Deep Learning Specialist” certification when you complete all five modules
     
    • Have an interest in Deep Learning?
    • Join us if you are able to read and follow codes
    • This module is compulsory before you take the advanced modules
     
    • You will need to take module 1 before this module
     
    • You will need to take module 1 before this module
     
    • You will need to take module 1 AND module 2 OR 3 before this module
     
    • You will need to take module 1, 2 AND 3 before this module
    • Attain a “Deep Learning Specialist” certification when you complete all five modules
     

About this module:

One of the core skills in Natural Language Processing (NLP) is reliably detecting entities and classifying individual words according to their parts of speech. We will look at how Named Entity Recognition (NER) works and how Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTMs) are used for tasks like this and many others in NLP.

Another common technique of Deep Learning in NLP is the use of word and character vector embeddings. We will cover well-known models as Word2Vec and GLoVE, how they are created, their unique properties and how you can use them to improve the accuracy of Natural Language in terms of understanding problems and applications.

We will also cover some of the recent developments in using transfer learning for text related problems and language modelling. These led to some of the recent state-of-the-art results for text classification problems like sentiment analysis and many more. This section will cover papers from ULMFIT , ELMo and OpenAI’s most recent Transformer model.

One of the biggest applications in Natural Language currently is the creation of chatbots and dialog systems. Thus, in this module you will discover how various types of chatbots work, the key technologies behind them and systems like Google’s DialogFlow and Duplex.

We will also look at applications such as the Neural Machine Translation. You will learn the recent developments and models that use these techniques, and various types of attention mechanisms that dramatically increased the quality of translation systems.

Beyond just text, this module will also cover time series predictions and how you can use techniques from the text-based models to make predictions on sequences. This opens up the range of applications to include financial time-series, continuous IoT readings, machinery failure prediction, website optimisation and trip planning.

As with all the other Deep Learning Developer modules you will have the opportunity to build multiple models yourself. These include your main project which gives you the ability to take these new skills and apply them to your field of work or interest.

In this course, participants will learn:

  • Text Classification Models and to build a text classifier
  • To build a Named Entity Recogniser (NER) system
  • About Sequence to Sequence models
  • To build NLP models from scratch
  • To build a Chatbot ML system
  • To build a language model

Recommended Prerequisites:

  • Must have attended Module 1: Deep Learning Jump-start Workshop
  • Ability to read and follow the code - We will send out some videos to help you with Python syntax specifically before the course begins.
  • Attendees MUST bring their own laptops

Interested but unable to make it on this date? Leave your details below and we will contact you for the next run.

Day 1

08:45am – 09:00am: Registration
09:00am – 10:45am: Recurrent Neural Networks (RNNs) Recap Part 1

  • RNNs
  • Long Short-Term Memory (LSTMs)
  • Word embeddings: Word2Vec, GloVE
  • Basic Char RNNs
  • Word RNNs
  • Build LSTM networks

10:45am – 11:00am: Tea Break
11:00am – 12:30pm: RNNs Recap Part 2
12:30pm – 1:30pm: Lunch
1:30pm – 3:00pm: Natural Language Processing (NLP) Part 1

  • Text classification models
  • BiDirectional LSTMs
  • Building a Named Entity Recogniser (NER) system
  • Sentiment analysis
  • Building a text classifier
  • Personal text project
  • Major Project Week 1

3:00pm – 3:15pm: Tea Break
3:15pm – 4:45pm: NLP Part 2
4:45pm – 5:15pm: Personal text project

  • Ideas on projects to do
  • Q&A on ‘doable projects’
  • Homework: What to bring to the next session

5:15pm – 5:30pm: Closing comments and questions

Day 2

08:45am – 09:00am: Registration
09:00am – 10:45am: Seq2Seq and CNNs for text
Frameworks: TensorFlow, Keras, PyTorch

  • Sequence to sequence models
  • Convolutions for text networks
  • Clustering
  • Seq2Seq Chatbot

10:45am– 11:00am: Tea Break
11:00am – 12:30pm: Project Clinic 1
Project questions and general follow up

12:30pm – 1:30pm: Lunch
1:30pm – 3:15pm: Time Series

  • Sequence to sequence models
  • Convolutions for text networks
  • Clustering
  • Seq2Seq Chatbot

3:15pm – 3:30pm: Tea Break
3:30pm – 4:30pm: Project Clinic 2
Project questions and general follow up

4:30pm – 5:00pm: Closing comments and questions

Participants will be given two weeks to complete their online learning and individual project.

Online Learning 

  • Building NLP models from scratch
  • NLP pipelines
  • Guide to using Spacy
  • Building a Chatbot Machine Learning system
  • Building a language model

Assessments
Participants must fulfil the criteria stated below to pass and complete the course.

1.    Online Tests: Participants are required to score an minimum grade of more than 75% 

2.    Project: Participants are required to present, and achieve a pass on project that demonstrates the following:

  • The ability to use or create a data processing pipeline that gets data in the correct format for running in a Deep Learning model
  • The ability to create a model from scratch or use transfer learning to create a Deep Learning model
  • The ability to train that model and get results.
  • The ability to evaluate the model on held out data

Funding Support

CITREP+ is a programme under the TechSkills Accelerator (TeSA) – an initiative of SkillsFuture, driven by Infocomm Media Development Authority (IMDA).


*Please see the section below on ‘Guide for CITREP+ funding eligibility and self-application process’

Funding Amount: 

  • CITREP+ covers up to 90% of your nett payable course fee depending on eligibility for professionals

Please note: funding is capped at $3,000 per course application

  • CITREP+ covers up to 100% funding of your nett payable course fee for eligible students / full-time National Servicemen (NSF)

Please note: funding is capped at $2,500 per course application

Funding Eligibility: 

  • Singaporean / PR
  • Meets course admission criteria
  • Sponsoring organisation must be registered or incorporated in Singapore (only for individuals sponsored by organisations)

Please note: 

  • Employees of local government agencies and Institutes of Higher Learning (IHLs) will qualify for CITREP+ under the self-sponsored category
  • Sponsoring SMEs organisation who wish to apply for up to 90% funding support for course must meet SME status as defined here

Claim Conditions: 

  • Meet the minimum attendance (75%)
  • Complete and pass all assessments and / or projects

Guide for CITREP+ funding eligibility and self-application process:

For more information on CITREP+ eligibility criteria and application procedure, please click here

In partnership with:Driven by:

  

In partnership with employers to support employability:

For enquiries, please send an email to learning@sginnovate.com


Dr Martin Andrews
Martin has over 20 years’ experience in Machine Learning and has used it to solve problems in financial modelling and has created AI automation for companies. His current area of focus and speciality is in natural language processing and understanding. In 2017, Google appointed Martin as one of the first 12 Google Developer Experts for Machine Learning. Martin is also one of the co-founders of Red Dragon AI. 


Sam Witteveen
Sam has used Machine Learning and Deep Learning in building multiple tech start-ups, including a children’s educational app provider which has over 4 million users worldwide. His current focus is AI for conversational agents to allow humans to interact easier and faster with computers. In 2017, Google appointed Sam as one of the first 12 Google Developer Experts for Machine Learning in the world. Sam is also one of the co-founders of Red Dragon AI. 

Topics: Artificial Intelligence / Deep Learning / Machine Learning / Robotics