This is an old revision of the document!

Notes on NLP

Papers / Websites

Attention Is All You Need (original paper): https://arxiv.org/abs/1706.03762
Distilling the Knowledge in a Neural Network: https://arxiv.org/abs/1503.02531
Well-Read Students Learn Better: On the Importance of Pre-training Compact Models: https://arxiv.org/abs/1908.08962

TensorFlow Hub: https://tfhub.dev/
Google Research, BERT - Smaller Models on Git: https://github.com/google-research/bert
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding: https://arxiv.org/pdf/1810.04805.pdf

Getting meaning from text: self-attention step-by-step video (Romain Futrzynski):
https://peltarion.com/blog/data-science/self-attention-video
The Illustrated Transformer (Jay Alammar):
http://jalammar.github.io/illustrated-transformer/
Paper Dissected: “Attention is All You Need” Explained:
https://mlexplained.com/2017/12/29/attention-is-all-you-need-explained/
Speech and Language Processing, Dan Jurafsky and James H. Martin (3rd ed. draft):
https://web.stanford.edu/~jurafsky/slp3/

RNN W3L08 - Attention Model (Andrew Ng): https://www.youtube.com/watch?v=FMXUkEbjf9k&feature=youtu.be
Attention is all you need; Attentional Neural Network Models | Łukasz Kaiser | Masterclass: https://www.youtube.com/watch?v=rBCqOTEfxvg&feature=youtu.be
Transformer Neural Networks - EXPLAINED! (Attention is all you need) (CodeEmporium): https://www.youtube.com/watch?v=TQQlZhbC5ps&feature=youtu.be
Attention in Neural Networks (CodeEmporium):
https://www.youtube.com/watch?v=W2rWgXJBZhU&t
[Transformer] Attention Is All You Need | AISC Foundational (Joseph Palermo (Dessa)):
https://www.youtube.com/watch?v=S0KakHcj_rs&feature=youtu.be
Ivan Bilan: Understanding and Applying Self-Attention for NLP | PyData Berlin 2018: https://www.youtube.com/watch?v=OYygPG4d9H0&feature=youtu.be
Transformer (Attention is all you need)(Minsuk Heo):
https://www.youtube.com/watch?v=z1xs9jdZnuY&feature=youtu.be
Self-attention step-by-step | How to get meaning from text | Peltarion Platform (Romain Futrzynski):
https://www.youtube.com/watch?v=-9vVhYEXeyQ&feature=emb_logo