Lecture 12-1 - Recurrent Neural Networks and Transformers

MIT Introduction to Deep Learning 6.S191: Lecture 2
Recurrent Neural Networks
Lecturer: Ava Soleimany
January 2022

For all lectures, slides, and lab materials: http://introtodeeplearning.com

Lecture Outline
0:00​ - Introduction
1:59​ - Sequence modeling
4:16​ - Neurons with recurrence
10:09 - Recurrent neural networks
11:42​ - RNN intuition
14:44​ - Unfolding RNNs
16:43 - RNNs from scratch
19:49 - Design criteria for sequential modeling
21:00 - Word prediction example
27:49​ - Backpropagation through time
30:02 - Gradient issues
33:53​ - Long short term memory (LSTM)
35:35​ - RNN applications
40:22 - Attention fundamentals
43:12 - Intuition of attention
44:53 - Attention and search relationship
47:16 - Learning attention with neural networks
54:52 - Scaling attention and applications
56:09 - Summary
Subscribe to stay up to date with new deep learning lectures at MIT, or follow us @MITDeepLearning on Twitter and Instagram to stay fully-connected!!