Finetuning Deep Learning Model for Review Rating Prediction

Abstract

It has been proven that the fact of reviewing websites has an enormous impact on the shopping behavior of customer. Usually, the system generates a star rating out of 5 for each online review based on the text input by the user. This is called the review rating prediction problem where the rating star is predicted for a given product or service based on the review text. This issue has become widely known and discussed in the field of deep learning and Natural Language Processing. Researchers have developed this domain interestingly especially with the emergence of the concept of transfer learning. XLNet is considered among the main pretrained models available through the transformers library. It is used for text classification and can be fine-tuned on downstream tasks. This article presents a study of the literature concerning these concepts. Later on, it presented a fine-tuning approach of the XLNet algorithm through two main phases based on the concept of transfer learning. To prove the effectiveness of this approach, experiments were done on the Yelp Dataset. Noticeably, the classification task using the new model has achieved an interesting result of 77%. It outperformed the XLNet's SOTA accuracy that was 70%.