Inside Transformers and BERT with PyTorch

Suman Debnath (~debnsuma)


The transformer is one of the most popular state-of-the-art deep (SOTA) learning architectures that is mostly used for natural language processing (NLP) tasks. Ever since the advent of the transformer, it has replaced RNN and LSTM for various tasks. The advent of the transformer created a major breakthrough in the field of NLP and also paved the way for new revolutionary architectures such as BERT.

In this session we will dive deep into Transformers, and shall understand how it uses the encoder-decoder architecture for a language translation task. We will learn how transformers overcome one of the major challenges which we had in recurrent models like RNN/LSTM in capturing the long-term dependency. Later we will learn about BERT and see how it differs from other embedding models. We will fine-tune a BERT model for one application using Amazon SageMaker using PyTorch


Basics of Machine Learning Basics of RNN/LSTM(Optional) Basics of Pythons

Video URL:

Speaker Info:

Suman Debnath is a Principal Developer Advocate at Amazon Web Services based in India. He tries to simplify the intricacies of AWS cloud services to developers, aids them to unravel its optimum possibilities and obtain its utmost usage into their application. His key focus areas are: Python, Machine learning, Data Analytics and Storage.

Speaker Links:

Section: Data Science, Machine Learning and AI
Type: Workshop
Target Audience: Advanced
Last Updated: