Inside Transformers and BERT with PyTorch

Suman Debnath (~debnsuma)


Description:

The transformer is one of the most popular state-of-the-art deep (SOTA) learning architectures that is mostly used for natural language processing (NLP) tasks. Ever since the advent of the transformer, it has replaced RNN and LSTM for various tasks. The advent of the transformer created a major breakthrough in the field of NLP and also paved the way for new revolutionary architectures such as BERT.

In this session we will dive deep into Transformers, and shall understand how it uses the encoder-decoder architecture for a language translation task. We will learn how transformers overcome one of the major challenges which we had in recurrent models like RNN/LSTM in capturing the long-term dependency. Later we will learn about BERT and see how it differs from other embedding models. We will fine-tune a BERT model for one application using Amazon SageMaker using PyTorch

Prerequisites:

Basics of Machine Learning Basics of RNN/LSTM(Optional) Basics of Pythons

Video URL:

https://www.youtube.com/watch?v=7CHRwMvVUgM&t=1327s

Speaker Info:

https://aws.amazon.com/developer/community/evangelists/suman-debnath/

Suman Debnath is a Principal Developer Advocate at Amazon Web Services based in India. He tries to simplify the intricacies of AWS cloud services to developers, aids them to unravel its optimum possibilities and obtain its utmost usage into their application. His key focus areas are: Python, Machine learning, Data Analytics and Storage.

Speaker Links:

https://resources.awscloud.com/ai-and-machine-learning/create-train-and-deploy-machine-learning-ml-models-using-familiar-sql-commands-level-200 https://www.youtube.com/watch?v=gTCIksJU_JU https://resources.awscloud.com/aws-modern-applications/modern-application-development-with-machine-learning-level-200 https://www.youtube.com/watch?v=0HGzfO7hseI&t=21s

Section: Data Science, Machine Learning and AI
Type: Workshop
Target Audience: Advanced
Last Updated: