Improving Sentiment Analysis using Negation Scope Detection and Negation Handling

Abstract

Negation is one of the challenges in sentiment analysis. Negation has an immense influence on how accurately text data can be classified. To find the accurate sentiments of users this research identifies that the presence of polarity-shifting words and the removal of negative stopwords leads to the flipped polarity of sentences. To resolve these challenges this research proposes a method for negation scope detection and handling in sentiment analysis. Negation cues (negative words) and non_cue words are classified using logistic regression. These negation cue and non_cue words in addition to lexical and syntactic features determine the negation scope (part of sentence affected by cue) using the Machine Learning (ML) approach i.e. Conditional Random Fields (CRF). Subsequently, in negation handling the sentiment intensity of each token in a sentence is established, and affected tokens are processed to determine the final polarity. It is revealed that sentiment analysis with negation handling and calculated polarity gives 3.61%, 2.64%, 2.7%, and 1.42% increase in accuracy for Logistic regression, Support Vector Machine, Decision Tree (DT), and Naive Bayes (NB) consecutively for Amazon food products dataset. Consecutively, 7.64%, 5%, and 1.44% improvement for Logistic Regression (LR), Support Vector Machine (SVM), and Naive Bayes for electronic dataset.