Find in Library

Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on how attention models have grown and changed for offline and streaming speech recognition in recurrent neural networks and Transformer-based systems.