BERT-BILSTM-ATTENTION NEURAL NETWORKS APPLICATION METHOD TO DETERMINE THE AUTHOR’S EMOTIONAL ATTITUDE TO THE TEXT

Yin S., Afanasyev G. I., Kalistratov A. P.

In this paper, a text sentiment analysis method based on the BERT-BiLSTM-Attention model is proposed. During preparation, the text is encoded using the BERT model to obtain a semantic representation of the text. Then, the encoded text is uploaded into the BiLSTM model to obtain deeper semantic information. Finally, the output of the BiLSTM model is weighted and averaged by the Attention mechanism to obtain the final sentiment analysis results. Experimenting on the IMDB dataset, the model proposed in this paper achieved an accuracy rate of 90%, providing better performance than other deep learning-based sentiment analysis methods.

Keywords:sentiment analysis, BERT-BiLSTM-Attention, Attention mechanism, deep learning, neural networks