For our project, my partner Amne Salame and I chose to focus on the article "Attention Is All You Need" by Vaswani et al. This paper introduces the Transformer model, which has become very important in NLP. The model uses a self-attention mechanism that helps process text more effectively than older methods like RNNs or LSTMs.
In the project, we plan to explain how the Transformer works, including self-attention, multi-head attention, and positional encoding. We’ll also talk about how this model is used in real-life applications like machine translation, text summarization, and text generation.
Source Material:
Title: Attention Is All You Need
Authors: Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I.
Published In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017)
Year of Publication: 2017
Rejected. Old and was presented in previous semesters.