Machine Learning for Open-ended Concept Map Proposition Assessment: Impact of Length on Accuracy
DOI:
https://doi.org/10.30812/matrik.v25i1.5044Keywords:
Concept Map, Open-ended, Classification, Text Representation, Machine LearningAbstract
Open-ended concept maps allow learners to freely connect concepts, enriching understanding by linking new and prior knowledge. However, manually assessing proposition quality is time-consuming and subjective. This study proposes an automatic classification model for proposition quality assessment using term frequency–inverse document frequency (TF-IDF), a text representation method based on word frequency, and several machine learning algorithms. Two datasets were used are Relational Database with an average 5 words per proposition and Cybersecurity Authentication with an average 10 words per proposition. Comparative experiments with Support Vector Machine (SVM), a supervised classification algorithm, K-Nearest Neighbor, Random Forest, and Long Short-Term Memory (LSTM), a recurrent neural network for sequence data, revealed that SVM with RBF kernel achieved the highest performance on shorter propositions 87% accuracy, Cohen’s Kappa 0.76, while LSTM showed greater strength in handling longer propositions 85% accuracy, Cohen’s Kappa 0.69. These findings suggest that proposition length influences model effectiveness. The proposed approach can reduce the burden of manual assessment, increase the objectivity of evaluation, and support more efficient implementation of concept maps in education.
Downloads
References
[1] D. Dwi Prasetya, T. Widiyaningtyas, and T. Hirashima, “Interrelatedness patterns of knowledge representation in extension concept mapping,” Res Pract Technol Enhanc Learn, vol. 20, no. 9, pp. 1–18, 2025, doi: https://doi.org/10.58459/rptel.2025.20009.
[2] D. D. Prasetya and T. Hirashima, “Associated Patterns in Open-Ended Concept Maps within E-Learning,” Knowledge Engineering and Data Science, vol. 5, no. 2, p. 179, Dec. 2022, doi: 10.17977/um018v5i22022p179-187.
[3] Y. Cooper and E. Zimmerman, “Concept Mapping: A Practical Process for Understanding and Conducting Art Education Research and Practice,” Art Education, vol. 73, no. 2, pp. 24–32, Mar. 2020, doi: 10.1080/00043125.2019.1695478.
[4] R. Ye. Bulyk and O. V. Kushniryk, “Use of Visualization Tools In The Educational Process,” Clin Exp Pathol, vol. 19, no. 3, pp. 155–160, Dec. 2020, doi: 10.24061/1727-4338.xix.3.73.2020.22.
[5] A. Guiral and M. Pifarré, “Digital Cognitive Maps for Scientific Knowledge Construction in Initial Teacher Education,” RIED-Revista Iberoamericana de Educacion a Distancia, vol. 26, no. 2, pp. 89–109, Jul. 2023, doi: 10.5944/ried.26.2.36067.
[6] K. E. de Ries, H. Schaap, A. M. M. J. A. P. van Loon, M. M. H. Kral, and P. C. Meijer, “A literature review of open-ended concept maps as a research instrument to study knowledge and learning,” Qual Quant, vol. 56, no. 1, pp. 73–107, Feb. 2022, doi: 10.1007/s11135-021-01113-x.
[7] R. F. Kusumadewi and I. Kusmaryono, “Concept maps as dynamic tools to increase students’ understanding of knowledge and creative thinking,” Premiere Educandum : Jurnal Pendidikan Dasar dan Pembelajaran, vol. 12, no. 1, p. 12, Jun. 2022, doi: 10.25273/pe.v12i1.11745.
[8] K. A. Kroeze, S. M. Van Den Berg, B. P. Veldkamp, and T. De Jong, “Automated Assessment of and Feedback on Concept Maps during Inquiry Learning,” IEEE Transactions on Learning Technologies, vol. 14, no. 4, pp. 460–473, Aug. 2021, doi: 10.1109/TLT.2021.3103331.
[9] D. Dwi Prasetya, T. Hirashima, and Y. Hayashi, “Study on Extended Scratch-Build Concept Map to Enhance Students’ Understanding and Promote Quality of Knowledge Structure,” IJACSA) International Journal of Advanced Computer Science and Applications, vol. 11, no. 4, pp. 144–153, 2020, doi: 10.14569/IJACSA.2020.0110420.
[10] T. A. Assegie, A. O. Salau, G. Chhabra, K. Kaushik, and S. L. Braide, “Evaluation of Random Forest and Support Vector Machine Models in Educational Data Mining,” in Proceedings - 2nd International Conference on Advancement in Computation and Computer Technologies, InCACCT 2024, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 131–135. doi: 10.1109/InCACCT61598.2024.10551110.
[11] M. Sheykhmousa, M. Mahdianpari, H. Ghanbari, F. Mohammadimanesh, P. Ghamisi, and S. Homayouni, “Support Vector Machine Versus Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review,” 2020, Institute of Electrical and Electronics Engineers Inc. doi: 10.1109/JSTARS.2020.3026724.
[12] H. Choi, H. Lee, and M. Lee, “Optimal Knowledge Component Extracting Model for Knowledge-Concept Graph Completion in Education,” IEEE Access, vol. 11, pp. 15002–15013, 2023, doi: 10.1109/ACCESS.2023.3244614.
[13] S. V. Tytenko, “Concept Maps, Their Application Types And Methods In Information And Learning Systems,” KPI Science News, vol. 2020, no. 4, pp. 70–78, Mar. 2020, doi: 10.20535/kpisn.2020.4.227090.
[14] F. Sciarrone and M. Temperini, “A Sentence-Embedding-Based Dashboard to Support Teacher Analysis of Learner Concept Maps,” Electronics (Switzerland), vol. 13, no. 9, pp. 1–9, May 2024, doi: 10.3390/electronics13091756.
[15] P. Huang, S. Lin, J. Yuan, and H. Chen, “Course Achievement Evaluation Using Concept Map in Traditional Learning,” J Phys Conf Ser, vol. 1624, no. 5, pp. 1–5, Nov. 2020, doi: 10.1088/1742-6596/1624/5/052014.
[16] S. LOIZOU, N. Nicolaou, B. A. Pincus, A. Papageorgiou, and P. McCrorie, “Concept maps as a novel assessment tool in medical education,” MedEdPublish, vol. 12, no. 21, pp. 1–25, Mar. 2022, doi: 10.12688/mep.19036.1.
[17] D. D. Prasetya, A. Pinandito, Y. Hayashi, and T. Hirashima, “Analysis of quality of knowledge structure and students’ perceptions in extension concept mapping,” Res Pract Technol Enhanc Learn, vol. 17, no. 1, pp. 14–25, Dec. 2022, doi: 10.1186/s41039-022-00189-9.
[18] S. Bhatia, S. Bhatia, and I. Ahmed, “Automated waterloo rubric for concept map grading,” IEEE Access, vol. 9, pp. 148590–148598, 2021, doi: 10.1109/ACCESS.2021.3124672.
[19] S. Xia, P. Zhan, K. K. H. Chan, and L. Wang, “Assessing concept mapping competence using item expansion-based diagnostic classification analysis,” J Res Sci Teach, vol. 61, no. 7, pp. 1516–1542, Sep. 2024, doi: 10.1002/tea.21897.
[20] A. Caputo, D. Monterosso, and E. Sorrentino, “The use of concept maps as an assessment tool in students’ risk education about occupational safety and health,” Ann Ist Super Sanita, vol. 58, no. 3, pp. 172–176, 2022, doi: 10.4415/ANN_22_03_05.
[21] S. Sahay Bisarya, A. Shukla, and S. Kumar, “Data Preparation in Context of Social Sciences Research,” International Journal for Multidisciplinary Research, vol. 5, no. 3, pp. 1–10, Jun. 2023, doi: https://doi.org/gsdk6t.
[22] Yuyun, A. D. Latief, T. Sampurno, Hazriani, A. O. Arisha, and Mushaf, “Next Sentence Prediction: The Impact of Preprocessing Techniques in Deep Learning,” in Proceedings - 2023 10th International Conference on Computer, Control, Informatics and its Applications: Exploring the Power of Data: Leveraging Information to Drive Digital Innovation, IC3INA 2023, Institute of Electrical and Electronics Engineers Inc., 2023, pp. 274–278. doi: 10.1109/IC3INA60834.2023.10285805.
[23] T. Mustaqim, K. Umam, and M. A. Muslim, “Twitter text mining for sentiment analysis on government’s response to forest fires with vader lexicon polarity detection and k-nearest neighbor algorithm,” in Journal of Physics: Conference Series, Institute of Physics Publishing, Jul. 2020. doi: 10.1088/1742-6596/1567/3/032024.
[24] M. Sergii V. and N. Oleksandr V., “Data preprocessing and tokenization techniquesfortechnical Ukrainian texts,” Applied Aspects of Information Technology, vol. 6, no. 3, pp. 318–326, Sep. 2023, doi: 10.15276/aait.06.2023.22.
[25] V. A. Kozhevnikov and E. S. Pankratova, “Research Of Text Pre-Processing Methods For Preparing Data In Russian For Machine Learning.,” Theoretical & Applied Science, vol. 84, no. 4, pp. 313–320, 2020, doi: 10.15863/TAS.
[26] Z. Abidin, A. Junaidi, and Wamiliana, “Text Stemming and Lemmatization of Regional Languages in Indonesia: A Systematic Literature Review,” Journal of Information Systems Engineering and Business Intelligence, vol. 10, no. 2, pp. 217–231, Jun. 2024, doi: 10.20473/jisebi.10.2.217-231.
[27] T. Hevianto Saputro and A. Hermawan, “The Accuracy Improvement of Text Mining Classification on Hospital Review through The Alteration in The Preprocessing Stage,” International Journal of Computer and Information Technology, vol. 10, no. 4, pp. 2279–0764, 2021, doi: https://doi.org/10.24203/ijcit.v10i4.138.
[28] H. D. Abubakar and M. Umar, “Sentiment Classification: Review of Text Vectorization Methods: Bag of Words, Tf-Idf, Word2vec and Doc2vec,” SLU Journal of Science and Technology, vol. 4, no. 1 & 2, pp. 27–33, Aug. 2022, doi: 10.56471/slujst.v4i.266.
[29] H. Zhou, “Research of Text Classification Based on TF-IDF and CNN-LSTM,” J Phys Conf Ser, vol. 2171, no. 1, pp. 1–8, Jan. 2022, doi: 10.1088/1742-6596/2171/1/012021.
[30] J. Zhou, Z. Ye, S. Zhang, Z. Geng, N. Han, and T. Yang, “Investigating response behavior through TF-IDF and Word2vec text analysis: A case study of PISA 2012 problem-solving process data,” Heliyon, vol. 10, no. 16, pp. 7–18, Aug. 2024, doi: 10.1016/j.heliyon.2024.e35945.
[31] V. Chaurasia and S. Pal, “Applications of Machine Learning Techniques to Predict Diagnostic Breast Cancer,” SN Comput Sci, vol. 12, no. 5, pp. 2–10, Sep. 2020, doi: 10.1007/s42979-020-00296-8.
[32] M. Yalsavar, P. Karimaghaee, A. Sheikh-Akbari, M. H. Khooban, J. Dehmeshki, and S. Al-Majeed, “Kernel Parameter Optimization for Support Vector Machine Based on Sliding Mode Control,” IEEE Access, vol. 10, pp. 17003–17017, 2022, doi: 10.1109/ACCESS.2022.3150001.
[33] U. G. Inyang, F. F. Ijebu, F. B. Osang, A. A. Afolorunso, S. S. Udoh, and I. J. Eyoh, “A Dataset-Driven Parameter Tuning Approach for Enhanced K-Nearest Neighbour Algorithm Performance,” Int J Adv Sci Eng Inf Technol, vol. 13, no. 1, pp. 380–391, 2023, doi: 10.18517/ijaseit.13.1.16706.
[34] A. F. M. Amidon, Z. M. Yusoff, N. Ismail, A. A. Almisreb, and M. N. Taib, “KNN Euclidean Distance Model Performance on Aquilaria Malaccensis Oil Qualities,” Journal of Advanced Research in Applied Sciences and Engineering Technology, vol. 48, no. 2, pp. 16–28, Jun. 2025, doi: 10.37934/araset.48.2.1628.
[35] H. Jagath Prasad and S. R. Marjorie, “Optimized k-nearest neighbours classifier based prediction of epileptic seizures,” Bulletin of Electrical Engineering and Informatics, vol. 13, no. 4, pp. 2442–2455, Aug. 2024, doi: 10.11591/eei.v13i4.6598.
[36] T. J. Angula and V. Hashiyana, “Detection of Structured Query Language Injection Attacks Using Machine Learning Techniques,” International Journal of Computer Science and Information Technology, vol. 15, no. 4, pp. 13–26, Aug. 2023, doi: 10.5121/ijcsit.2023.15402.
[37] A. Sharma and S. Kumar, “Ontology-based semantic retrieval of documents using Word2vec model,” Data Knowl Eng, vol. 144, no. 102110, pp. 2–15, Mar. 2023, doi: 10.1016/j.datak.2022.102110.
[38] D. Li, G. Li, S. Li, and A. Bang, “Classification Prediction of Lung Cancer Based on Machine Learning Method,” International Journal of Healthcare Information Systems and Informatics, vol. 19, no. 1, pp. 4–9, 2023, doi: 10.4018/IJHISI.333631.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Reo Wicaksono, Didik Dwi Prasetya, Ilham Ari Elbaith Zaeni, Nadindra Dwi Ariyanta, Tsukasa Hirashima
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
Similar Articles
- Michael Michael, Frenky Tanoto, Eric Wibowo, Frenky Lutan, Abdi Dharma, Pengenalan Plat Kendaraan Bermotor dengan Menggunakan Metode Template Matching dan Deep Belief Network , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 19 No. 1 (2019)
- Nur Fitrianingsih Hasan, Muhammad Salis Amin Iribaram, Digitalisasi Kamus Bahasa Daerah Papua Menggunakan Metode Rapid Application Development , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 21 No. 3 (2022)
- Soleh Ardiansyah, Adani Setiorini, Lovinta Happy Atrinawati, Tegar Palyus Fiqar, Perancangan Arsitektur Sistem dan Teknologi Informasi Menggunakan Togaf ADM (Studi Kasus Dinas Perhubungan Kota Balikpapan) , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 19 No. 1 (2019)
- Fathurrahmani Fathurrahmani, Wiwik Kusrini, Khairul Anwar Hafizd, Arif Supriyanto, Penerapan Sistem Tertanam untuk Monitoring Kandang Ayam Broiler , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 19 No. 1 (2019)
- Guntoro Guntoro, Lisnawita Lisnawita, Loneli Costaner, Optimizing Random Forest for IoT Cyberattack Detection UsingSMOTE: A Study on CIC-IoT2023 Dataset , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 25 No. 1 (2025)
- Muchammad Ismail Hamzah, DESAIN DAN IMPLEMENTASI WEB SMP ISLAM NGEBRUK MENGGUNAKAN PHP DAN MYSQL , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 16 No. 2 (2017)
- Rakandhiya Daanii Rachmanto, Ahmad Naufal Labiib Nabhaan, Arief Setyanto, Deep Learning Model Compression Techniques Performance on Edge Devices , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 3 (2024)
- Mudafiq Riyan Pratama, Muhammad Yunus, Sistem Deteksi Struktur Kalimat Bahasa Arab Menggunakan Algoritma Light Stemming , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 19 No. 1 (2019)
- Uswatun Hasanah, Neny Sulistianingsih, PEMODELAN SISTEM PENJADWALAN PRAKTIKUM LABORATORIUM MENGGUNAKAN ALJABAR MAXPLUS (STUDI KASUS DI STMIK BUMIGORA MATARAM) , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 15 No. 1 (2015)
- Deny Jollyta, Prihandoko Prihandoko, Dadang Priyanto, Alyauma Hajjah, Yulvia Nora Marlim, Comparison of Distance Measurements Based on k-Numbers and Its Influence to Clustering , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 1 (2023)
You may also start an advanced similarity search for this article.
Most read articles by the same author(s)
- Wahyu Styo Pratama, Didik Dwi Prasetya, Triyanna Widyaningtyas, Muhammad Zaki Wiryawan, Lalu Ganda Rady Putra, Tsukasa Hirashima, Performance Evaluation of Artificial Intelligence Models for Classification in Concept Map Quality Assessment , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- F.ti Ayyu Sayyidul Laily, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Revealing Interaction Patterns in Concept Map Construction Using Deep Learning and Machine Learning Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Sucipto Sucipto, Didik Dwi Prasetya, Triyanna Widiyaningtyas, Educational Data Mining: Multiple Choice Question Classification in Vocational School , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 2 (2024)
- Reni Fatrisna Salsabila, Didik Dwi Prasetya, Triyanna Widyaningtyas, Tsukasa Hirashima, Comparison of Text Representation for Clustering Student Concept Maps , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Muhammad Zaki Wiryawan, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Wahyu Styo Pratama, Lalu Ganda Rady Putra, Enhancing Semantic Similarity in Concept Maps Using LargeLanguage Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- Nadindra Dwi Ariyanta, Didik Dwi Prasetya, Ilham Ari Elbaith Zaeni, Tsukasa Hirashima, Reo Wicaksono, Assessing the Semantic Alignment in Multilingual Student-TeacherConcept Maps Using mBERT , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 25 No. 1 (2025)
.png)
