Comparative Analysis of Machine Learning and Deep Learning Models for Sentiment Analysis of Mobile Game Reviews

Iyus Rusmansyah

doi:10.65780/bima.v1i4.20

Authors

Iyus Rusmansyah Institut Citra Buana Indonesia Author

DOI:

https://doi.org/10.65780/bima.v1i4.20

Keywords:

Sentiment Analysis, Mobile Game Reviews, Machine Learning, Deep Learning, Imbalanced Dataset, NLP

Abstract

The rapid growth of mobile gaming has led to a substantial increase in user-generated reviews, providing valuable insights into user experience. However, many existing sentiment analysis studies on mobile game reviews rely on balanced datasets and pay limited attention to the challenges of multi-class classification under imbalanced conditions, particularly for minority classes such as neutral sentiment. In addition, limited studies systematically examine how class imbalance affects the comparative performance of Machine Learning and Deep Learning models within a unified experimental setting. This study evaluates the performance of Machine Learning and Deep Learning approaches for sentiment analysis using imbalanced mobile game review data. A dataset of 5000 reviews collected from the Google Play Store is categorized into three classes: positive, neutral, and negative. Light Gradient Boosting Machine (LightGBM) and Convolutional Neural Network (CNN) are used as representative models, with class weighting applied to address data imbalance. The findings show that CNN achieves slightly higher accuracy (68.20%) than LightGBM (66.40%), although both models show comparable performance in macro-average metrics. Both approaches experience difficulty in identifying the neutral class, reflecting the impact of class imbalance. These findings emphasize that class distribution plays a more critical role than model choice in real-world sentiment classification.

Downloads

Download data is not yet available.

References

[1] Z. Yejian and S. Takada, “Review Classification Based on Machine Learning: Classifying Game User Reviews,” IEEE Access, vol. 11, pp. 142447–142463, 2023, doi: 10.1109/ACCESS.2023.3342294.

[2] G. Mutanov, V. Karyukin, and Z. Mamykova, “Multi-Class Sentiment Analysis of Social Media Data with Machine Learning Algorithms,” Computers, Materials & Continua, vol. 69, no. 1, pp. 913–930, 2021, doi: 10.32604/cmc.2021.017827.

[3] Y. Yu, T. Dinh, F. Yu, and V.-N. Huynh, “Understanding Mobile Game Reviews Through Sentiment Analysis: A Case Study of PUBGm,” in Model and Data Engineering, vol. 14396, M. Mosbah, T. Kechadi, L. Bellatreche, and F. Gargouri, Eds., in Lecture Notes in Computer Science, vol. 14396. , Cham: Springer Nature Switzerland, 2024, pp. 102–115. doi: 10.1007/978-3-031-49333-1_8.

[4] Faculty of Computing and Information Technology, Tunku Abdul Rahman University College, Kuala Lumpur, Malaysia, J. Y. Tan, A. S. K. Chow, Faculty of Computing and Information Technology, Tunku Abdul Rahman University College, Kuala Lumpur, Malaysia, and Faculty of Computing and Information Technology, Tunku Abdul Rahman University College, Kuala Lumpur, Malaysia, “Sentiment Analysis on Game Reviews: A Comparative Study of Machine Learning Approaches,” in Conference Proceedings: International Conference on Digital Transformation and Applications (ICDXA 2021), Tunku Abdul Rahman University College, 2021, pp. 209–216. doi: 10.56453/icdxa.2021.1023.

[5] K. T. Wong, “The Data-Driven Myth and the Deceptive Futurity of ‘the World’s Fastest Growing Games Region’: Selling the Southeast Asian Games Market via Game Analytics,” Games and Culture, vol. 18, no. 1, pp. 42–61, Jan. 2023, doi: 10.1177/15554120221077731.

[6] J. Y. Tan, A. S. K. Chow, and C. W. Tan, “A Comparative Study of Machine Learning Algorithms for Sentiment Analysis of Game Reviews,” TJIEM, vol. 82, no. 3, Nov. 2022, doi: 10.54552/v82i3.101.

[7] M. Das, “A Comparative Study on TF-IDF feature Weighting Method and its Analysis using Unstructured Dataset”.

[8] B. Krawczyk, “Learning from imbalanced data: open challenges and future directions,” Prog Artif Intell, vol. 5, no. 4, pp. 221–232, Nov. 2016, doi: 10.1007/s13748-016-0094-0.

[9] N. VasfiSisi and M. R. F. Derakhshi, “Text Classification with Machine Learning Algorithms,” 2013.

[10] G. Mutanov, V. Karyukin, and Z. Mamykova, “Multi-Class Sentiment Analysis of Social Media Data with Machine Learning Algorithms,” Computers, Materials & Continua, vol. 69, no. 1, pp. 913–930, 2021, doi: 10.32604/cmc.2021.017827.

[11] M. Dwifebri Purbolaksono, “Sentiment Analysis of Game Review in Steam Platform using Random Forest,” ijoict, vol. 10, no. 2, pp. 161–169, Dec. 2024, doi: 10.21108/ijoict.v10i2.1007.

[12] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” jair, vol. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.

[13] D. I. Sumantiawan, J. E. Suseno, and W. A. Syafei, “Sentiment Analysis of Customer Reviews Using Support Vector Machine and Smote-Tomek Links For Identify Customer Satisfaction,” J. Sistem Info. Bisnis, vol. 13, no. 1, pp. 1–9, Jun. 2023, doi: 10.21456/vol13iss1pp1-9.

[14] J. Y. Tan, A. S. K. Chow, and C. W. Tan, “A Comparative Study of Machine Learning Algorithms for Sentiment Analysis of Game Reviews,” TJIEM, vol. 82, no. 3, Nov. 2022, doi: 10.54552/v82i3.101.

[15] M. I. U. Sarkar, A. Srinivasulu, A. Lal, R. Naidu, and M. M. Rahman, “Classification of toxic element accumulation in rice grains using optimized machine learning models: A comparative study,” Journal of Food Composition and Analysis, vol. 148, p. 108504, Dec. 2025, doi: 10.1016/j.jfca.2025.108504.

[16] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” jair, vol. 16, pp. 321–357, Jun. 2002, doi: 10.1613/jair.953.