A Comprehensive Framework for Prioritizing User Concerns in Mobile App Reviews Using Multi-Metric Scoring

Abstract

An increasing number of mobile app user reviews is a vital source on user concerns towards existing apps. These reviews help to optimize and improve the apps. Despite the recent introduction of effective user review analysis methods, analyzing user reviews still poses significant challenges for researchers. One of them is the overwhelming number of informative reviews make it difficulty extract and prioritize user concerns. This research proposes a novel framework to prioritize user concerns in mobile app reviews utilizing Natural Language Processing (NLP) techniques such as sentiment analysis, Latent Dirichlet Allocation (LDA), and word embedding. This comprehensive framework extracts and ranks user concerns and opinions related to user experience (UX) using a weighted scoring mechanism; multi-criteria prioritization formula. This formula includes four key metrics: Entropy score, Topic Prevalence score, Thumbsup count, and Sentiment score for the major topics identified in the reviews. The proposed framework was evaluated using user reviews from eight mobile apps across four popular categories: education, messaging, business, and shopping. A total of 869,731 reviews were scraped from the Play Store for this evaluation. To validate the proposed framework, its prioritization results were compared with a dataset prioritized by expert app developers. Spearman’s rank correlation was used to compare the prioritization trends and the average correlation was 0.7569. Additionally, the Mean Absolute Error (MAE) was 0.1724. These results show that the proposed prioritization framework aligns with the expert developers’ priorities with a marginal error. Furthermore, this framework is generalizable, as the evaluation included apps from diverse categories. This makes the proposed framework an effective and efficient tool for decision-making in patch, update or version releases in mobile apps, ensuring that critical user concerns are addressed promptly.