A Comparative Sentiment Analysis of Learning Management Systems Reviews Using Machine Learning within the SEMMA Framework
DOI:
https://doi.org/10.51278/bce.v6i3.2826Keywords:
Sentiment Analysis, Machine Learning, Learning Management System, Support Vector Machine, SEMMAAbstract
Mobile Learning Management Systems (LMS) have become essential in supporting digital learning by providing flexible access to academic resources and communication between students and educational institutions. User reviews published on the Google Play Store provide valuable feedback for evaluating service quality; however, their unstructured nature makes systematic analysis challenging. This study aims to analyze user sentiment toward the LMS Mobile application of Universitas XYZ and compare the performance of the Naïve Bayes and Support Vector Machine (SVM) algorithms in sentiment classification. A quantitative experimental approach was employed using 129 user reviews collected through web scraping from the Google Play Store between November 2020 and December 2025. The analysis followed the SEMMA (Sample, Explore, Modify, Model, Assess) framework and applied Natural Language Processing (NLP) techniques, including text cleaning, tokenization, case folding, stopword removal, and stemming. Textual data were transformed into numerical features using the Term Frequency–Inverse Document Frequency (TF-IDF) method before being classified with Naïve Bayes and SVM. Model performance was evaluated using confusion matrix metrics, including accuracy, precision, recall, and F1-score. The results indicate that negative sentiment dominates user reviews, primarily concerning login problems, application errors, and system performance. The Naïve Bayes model achieved a higher accuracy (0.69) than SVM (0.65), whereas SVM demonstrated more balanced classification performance across sentiment categories. These findings provide a data-driven basis for evaluating user feedback and supporting continuous improvement of digital learning services within higher education.
Keywords: Sentiment Analysis, Machine Learning, Learning Management System, Support Vector Machine, SEMMA.
References
Abdullah, D. M., & Abdulazeez, A. M. (2021). Machine Learning Applications based on SVM Classification: A Review. Qubahan Academic Journal, 1(2), 81–90. DOI: 10.48161/qaj.v1n2a50.
Ali, S., Fatima, F., Hussain, J., Qureshi, M. I., Fatima, S., & Zahoor, A. (2023). International Journal of Interactive Mobile Technologies (iJIM), 17(13), 4–21. DOI: 10.3991/ijim.v17i13.39785.
Aljouni, A., Alkhawaldeh, R. S., Al-Okaily, M. M., Al-Debei, M. M., & Alshurideh, M. (2024). Factors influencing student satisfaction with LMS mobile application: A TAM-based SEM analysis. International Journal of Interactive Mobile Technologies, 18(19). https://doi.org/10.3991/ijim.v18i19.50237
Andrade-Arenas, L., Rubio-Paucar, I., & Yactayo-Arias, C. (2024). Data mining for predictive analysis in gynecology: A focus on cervical health. International Journal of Electrical and Computer Engineering, 14(3), 2822–2833. https://doi.org/10.11591/ijece.v14i3.pp2822-2833
Cahyani, D. E., & Patasik, I. (2021). Performance comparison of TF-IDF and Word2Vec models for emotion text classification. Bulletin of Electrical Engineering and Informatics, 10(5), 2780– 2788. https://doi.org/10.11591/eei.v10i5.3157
Ganganwar, V., & Rajalakshmi, R. (2024). Employing synthetic data for addressing the class imbalance in aspect-based sentiment classification. Journal of Information and Telecommunication, 8(2), 167–188. DOI: 10.1080/24751839.2023.2270824.
Gardazi, N. M., Daud, A., Malik, M. K., Bukhari, A., Alsahfi, T., & Alshemaimri, B. (2025). BERT applications in natural language processing: A review. Artificial Intelligence Review, 58, 166. DOI: 10.1007/s10462-025-11162-5.
Gómez Palacios et al. (2017). A Comparative between CRISP-DM and SEMMA through the Construction of a MODIS Repository for Studies of Land Use and Cover Change. DOI: 10.25046/aj020376
Hokijuliandy, E., Napitupulu, H., & Firdaniza. (2023). Application of SVM and Chi-Square Feature Selection for Sentiment Analysis of Indonesia’s National Health Insurance Mobile Application. Mathematics, 11(17), 3765. DOI: 10.3390/math11173765.
Ikhsan, R. B., Prabowo, H., Yuniarty, Simamora, B., Ruan, X., & Kumar, V. (2022). Predicting students' use of mobile-learning management systems in Indonesia. Journal of Educators Online, 19(3). https://doi.org/10.9743/JEO.2022.19.3.5
Indriani, F., Nugroho, R. A., Faisal, M. R., & Kartini, D. (2024). Comparative evaluation of IndoBERT, IndoBERTweet, and mBERT for multilabel student feedback classification. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), 8(6), 748–757. DOI: 10.29207/resti.v8i6.6100.
Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31, 685–695. DOI: 10.1007/s12525-021-00475-2.
Jannah, N. Z. B., & Kusnawi. (2024). Comparison of Naïve Bayes and SVM in sentiment analysis of product reviews on marketplaces. Sinkron: Jurnal dan Penelitian Teknik Informatika, 8(2), 727–733. https://doi.org/10.33395/sinkron.v8i2.13559.
Johnson, J. M., & Khoshgoftaar, T. M. (2019). Survey on deep learning with class imbalance. Journal of Big Data, 6, 27. DOI: 10.1186/s40537-019-0192-5.
Kaisiang, C., & Shah, N. (2022). Comparison of Naive Bayes and SVM classification in grid-search hyperparameter tuned and non-hyperparameter tuned healthcare stock market sentiment analysis. IAENG International Journal of Computer Science, 49(4), 536–544.
Kurniawan, R., Wijaya, H. O. L., & Aprisusanti, R. P. (2024). Sentiment analysis of Google Play Store user reviews on digital population identity app using K-nearest neighbors. Jurnal SISFOKOM (Sistem Informasi dan Komputer), 13(2), 170–178. https://doi.org/10.1145/3569897
Mustaqim, I. Z., & Suryono, R. R. (2025). A systematic literature review of topic modeling techniques in user reviews. Journal of Information Systems Engineering and Business Intelligence, 11(2), 238–253. DOI: 10.20473/jisebi.11.2.238-253.
Ngulube, P., & Ncube, M. M. (2025). Leveraging learning analytics to improve the user experience of learning management systems in higher education institutions. Information, 16(5), 419. https://doi.org/10.3390/info16050419
Ohliati, J., & Abbas, B. S. (2019). Measuring Students Satisfaction in Using Learning Management System. International Journal of Emerging Technologies in Learning (iJET), 14(4), 180–189. DOI: 10.3991/ijet.v14.i04.9427.
Pambudi, S., Setiaji, P., & Triyanto, W. A. (2025). Sentiment analysis of Fizzo Novel application using Support Vector Machine and Naïve Bayes algorithm with SEMMA framework. Jurnal Teknik Informatika (JUTIF), 6(4), 1861–1880. https://doi.org/10.52436/1.jutif.2025.6.4.4875
Patil, R., Boit, S., Gudivada, V., & Nandigam, J. (2023). A Survey of Text Representation and Embedding Techniques in NLP. IEEE Access, 11, 36120–36146. DOI: 10.1109/ACCESS.2023.3266377.
Rahmad, F., Suryanto, Y., & Ramli, K. (2020). Performance comparison of anti-spam technology using confusion matrix classification. IOP Conference Series: Materials Science and Engineering, 879(1). https://doi.org/10.188/1757-899X/879/1/012076
Ressan, M. B., & Hassan, R. F. (2022). Naïve-Bayes family for sentiment analysis during COVID-19 pandemic and classification tweets. Indonesian Journal of Electrical Engineering and Computer Science, 28(1), 375–383. https://doi.org/10.11591/ijeecs.v28.i1.pp375-383
Rini, F., Yelfiza, & Pernanda, A. Y. (2023). New LMS Mobile Framework Based on Multiplatform: A Literature Review of Mobile LMS Theory, Design and Implementation. DOI: 10.47191/ijmra/v6-i6-54.
Sarker, I. H. (2021). Machine learning: Algorithms, real-world applications and research directions. SN Computer Science, 2(3), 160. http://doi.org/10.1007/s42979-021-00592-x
Sathyanarayanan, S., & Tantri, B. R. (2024). Confusion matrix-based performance evaluation metrics. African Journal of Biomedical Research, 27(4S), 4023–4031. https://doi.org/10.53555/AJBR.v27i4S.4345
Salur, M. U., & Aydin, I. (2020). A novel hybrid deep learning model for sentiment classification. IEEE Access, 8, 58080–58093. DOI: 10.1109/ACCESS.2020.2982538.
Setiawan, T., Liem, S., & Pribadi, M. R. (2024). Perbandingan algoritma SVM dan Naïve Bayes dalam analisis sentimen komentar TikTok pada produk skincare. Applied Information Technology and Computer Science (AICOMS), 3(2), 28–32. http://doi.org/10.58466/aicoms.v3i1.1523
Siddharth, L., Blessing, L., & Luo, J. (2022). Natural language processing in-and-for design research. Design Science, 8, e21. http://doi.org/10.1017/dsj.2022.16.
Siino, M., Tinnirello, I., & La Cascia, M. (2024). Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiers. Information Systems, 121, 102342. DOI: 10.1016/j.is.2023.102342.
Singgalen, Y. A. (2024). Distribution of Polarity Value between VADER and TextBlob in Sentiment Classification of Tourist Vlog Content Reviews. Journal of Information System Research (JOSH), 5(4), 1445–1454. DOI: 10.47065/josh.v5i4.5501.
Sinulingga, J. E. B., & Sitorus, H. C. K. (2024). Analisis sentimen masyarakat terhadap film horor Indonesia menggunakan metode SVM dan TF-IDF. Jurnal Manajemen Informatika (JAMIKA), 14(1), 42–53. http://doi.org/10.34010/jamika.v14i1.11946
Van Mechelen, M., et al. (2023). Emerging technologies in K–12 education: A future HCI research agenda. ACM Transactions on Computer-Human Interaction, 30(3), 1–40. http://doi.org/0.1145/3569897
Verbraaken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., & Rellermeyer, J. S. (2020). A survey on distributed machine learning. ACM Computing Surveys, 53(2), 1–33. https://doi.org/10.1145/3377454
Wati, R., Ernawati, S., & Rachmi, H. (2023). Pembobotan TF-IDF Menggunakan Naïve Bayes Pada Sentimen Masyarakat Mengenai Isu Kenaikan BIPIH. DOI: 10.34010/jamika.v13i1.9424.
Yelfiza, Rini, F., & Pernanda, A. Y. (2024). Measuring user satisfaction: Open source LMS mobile application as a learning technology. Journal of Advanced Research in Applied Sciences and Engineering Technology, 38(2), 176–185. http://doi.org/10.37934/araset.38.2.176185.
Yustanti, W., Iriawan, N., & Irhamah. (2023). Categorical encoder based performance comparison in pre-processing imbalanced multiclass classification. Jurnal Teknik Industri, 25(1), 39–54. https://doi.org/10.9744/jti.25.1.39-54
Zhang, Y., Li, Q., & Xin, Y. (2024). Research on eight machine learning algorithms applicability on different characteristics data sets in medical classification tasks. DOI: 10.3389/fncom.2024.1345575.
Zhou, J., Ye, Z., Zhang, S., Geng, Z., Han, N., & Yang, T. (2024). Investigating response behavior through TF-IDF and Word2vec text analysis: A case study of PISA 2012 problem-solving process data. DOI: 10.1016/j.heliyon.2024.e35945.
Zulfikar, W. B., Atmadja, A. R., & Pratama, S. F. (2023). Sentiment Analysis on Social Media Against Public Policy Using Multinomial Naive Bayes, 10(1), 25–34. DOI: 10.15294/sji.v10i1.39952.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Rahmat Rizki Hidayah, Vianny Utami
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
