Transformer

A deep learning Natural Language Processing model proposed in 2017 by a team of Google engineers, whose main innovation is to give a central role to the “attention” mechanism, which consists of weighting the relative importance of each word in relationship to others. This parallel (rather than sequential) treatment requires less computing and has made it possible to multiply the size of training sets and the number of parameters, contributing to the emergence of large language models.