Assessing the Semantic Alignment in Multilingual Student-TeacherConcept Maps Using mBERT
DOI:
https://doi.org/10.30812/matrik.v25i1.5046Keywords:
Concept Map, mBert, Multilingual, Open-ended, Semantic Similarity, TF-IDFAbstract
This study examines the effectiveness of mBERT (Multilingual Bidirectional Encoder Representations from Transformers) in assessing semantic alignment between student and teacher concept maps in multilingual educational contexts, comparing its performance with TF-IDF. Using datasets in both Indonesian and English, the study demonstrates that mBERT outperforms TF-IDF in capturing complex
semantic relationships, achieving 96% accuracy, 96% precision, 100% recall, and a 98% F1 score in the Indonesian dataset. In contrast, TF-IDF achieved higher precision (73%) and accuracy (79%) in the English dataset, where mBERT recorded 54% accuracy, 47% precision, but 90% recall. Semantic alignment was measured using cosine similarity to calculate the cosine of the angle between vectors
representing textual embeddings generated by both models. This method facilitates cross-linguistic semantic comparison, overcoming challenges related to word frequency and syntactic variations. While mBERT’s computational demands and the study’s limited linguistic scope suggest room for improvement, the findings highlight the potential for hybrid models and emphasize the transformative impact of AI-driven tools, such as mBERT, in fostering inclusive and effective multilingual education.
Downloads
References
[1] D. D. Prasetya, T. Widiyaningtyas, and T. Hirashima, “Interrelatedness patterns of knowledge representation in extension concept
mapping,” vol. 20, p. 009, May,2024, https://doi.org/10.58459/rptel.2025.20009.
[2] M. Konu KadiRhanogullari, “The Effect of Teaching with Concept Maps on Academic Success in Biology Teaching: A Meta-
Analysis Study,” no. 58, pp. 2781–2796, December, 2023, https://doi.org/10.53444/deubefd.1324169.
[3] C. G. M. Fine and E. M. Furtak, “A framework for science classroom assessment task design for emergent bilingual learners,”
vol. 104, no. 3, pp. 393–420, May, 2020, https://doi.org/10.1002/sce.21565.
[4] D. D. Prasetya, A. Pinandito, Y. Hayashi, and T. Hirashima, “Analysis of quality of knowledge structure and students’ perceptions
in extension concept mapping,” vol. 17, no. 1, p. 14, December, 2022, https://doi.org/10.1186/s41039-022-00189-9.
[5] J. M. Goodrich, L. Thayer, and S. Leiva, “Evaluating Achievement Gaps Between Monolingual and Multilingual Students,”
vol. 50, no. 7, pp. 429–441, October, 2021, https://doi.org/10.3102/0013189X21999043.
[6] S. Bhatia, S. Bhatia, and I. Ahmed, “AutomatedWaterloo Rubric for Concept Map Grading,” vol. 9, pp. 148 590–148 598, 2021,
https://doi.org/10.1109/ACCESS.2021.3124672.
[7] J. Nordmeyer, “From Testing to Teaching: Equity for Multilingual Learners in International Schools,” vol. 125, no. 7–8, pp.
247–275, August, 2023, https://doi.org/10.1177/01614681231194413.
[8] J. Mancilla-Martinez, J. K. Hwang, and M. H. Oh, “Assessment Selection for Multilingual Learners’ Reading Development,”
vol. 75, no. 3, pp. 351–362, November, 2021, https://doi.org/10.1002/trtr.2053.
[9] M. E. Flognfeldt, D. Tsagari, D. ˇ Surkalovi´c, and T. Tishakov, “The practice of assessing Norwegian and English language
proficiency in multilingual elementary school classrooms in Norway,” vol. 17, no. 5, pp. 519–540, October, 2020, https://doi.
org/10.1080/15434303.2020.1827409.
[10] M. Aparici, E. Rosado, and L. Tolchinsky, “Multilingual use assessment questionnaire: A proposal for assessing language and
literacy experience,” vol. 9, p. 1394727, May, 2024, https://doi.org/10.3389/fcomm.2024.1394727.
[11] N. Donmez Usta, E. ultay, and N. ultay, “Reading the Concept Map of Physics Teacher Candidates: A Case of Light,” vol. 31,
no. 1, pp. 14–21, March, 2020, https://doi.org/10.33828/sei.v31.i1.2.
[12] N. Fauziah, N. Izzati, and H. Handoko, “Development of Cooperative Integrated Reading and Composition Learning Model with
Mind Mapping Method to Improve Students’ Understanding of Mathematical Concepts,” vol. 1, no. 3, pp. 117–130, November,
2022, https://doi.org/10.58421/gehu.v1i3.27.
[13] H. L. Blake, “Intelligibility Enhancement via Telepractice During COVID-19 Restrictions,” vol. 5, no. 6, pp. 1797–1800,
December, 2020, https://doi.org/10.1044/2020 PERSP-20-00133.
[14] A. L. Ferrell, L. Soltero-Gonz´alez, and S. Kamioka, “Beyond English centrality: Integrating expansive conceptions of language
for literacy programming into IEPs,” vol. 9, p. 1347503, May, 2024, https://doi.org/10.3389/feduc.2024.1347503.
[15] D. Colla, E. Mensa, and D. P. Radicioni, “LessLex: Linking Multilingual Embeddings to SenSe Representations of LEXical
Items,” vol. 46, no. 2, pp. 289–333, June, 2020, https://doi.org/10.1162/coli a 00375.
[16] J. Heuzeroth and A. Budke, “The Effects of Multilinguality on the Development of Causal Speech Acts in the Geography
Classroom,” vol. 10, no. 11, p. 299, October, 2020, https://doi.org/10.3390/educsci10110299.
[17] F. A. S. Laily, D. D. Prasetya, A. N. Handayani, and T. Hirashima, “Revealing Interaction Patterns in Concept Map Construction
Using Deep Learning and Machine Learning Models,” vol. 24, no. 2, pp. 207–218, Februari, 2025, https://doi.org/10.30812/
matrik.v24i2.4641.
[18] S. Qin, L. Orchakova, Z.-Y. Liu, Y. Smirnova, and E. Tokareva, “Using the Learning Management System ”Modular Object-
Oriented Dynamic Learning Environment” in Multilingual Education,” vol. 17, no. 03, pp. 173–191, Februari, 2022, https:
//doi.org/10.3991/ijet.v17i03.25851.
[19] J. O. Uguru, “A Lexico-phonetic Comparison of Olukumi and Lukumi: A Procedure for Developing a Multilingual Dictionary,”
vol. 31, no. 1, May, 2021, https://doi.org/10.5788/31-1-1643.
[20] R. A. J. R. Peixoto, “Political Boundaries in Language Policies: A Discussion on Institutional Settings,” vol. 24, pp. e–1982–
4017–24–17, 2024, https://doi.org/10.1590/1982-4017-24-17.
[21] A. I. Anisimova, N. A. Safonova, M. Y. Dobrushyna, N. O. Lysenko, and I. H. Bezrodnykh, “Verbalization of the concept
language policy: Online research,” vol. 5, no. S4, pp. 1301–1311, November, 2021, https://doi.org/10.21744/lingcure.v5nS4.
1779.
[22] M. Perquin, S. Viswanathan, M. Vaillant, O. Risius, L. Huiart, J.-C. Schmit, N. J. Diederich, G. R. Fink, and J. Kukolja, “An
individualized functional magnetic resonance imaging protocol to assess semantic congruency effects on episodic memory in
an aging multilingual population,” vol. 14, p. 873376, July, 2022, https://doi.org/10.3389/fnagi.2022.873376.
[23] J. M. Giesinger, F. L. Loth, N. K. Aaronson, J. I. Arraras, G. Caocci, F. Efficace, M. Groenvold, M. Van Leeuwen, M. A.
Petersen, J. Ramage, K. A. Tomaszewski, T. Young, and B. Holzner, “Thresholds for clinical importance were established to
improve interpretation of the EORTC QLQ-C30 in clinical practice and research,” vol. 118, pp. 1–8, Februari, 2020, https:
//doi.org/10.1016/j.jclinepi.2019.10.003.
[24] K. Ro, J. Y. Kim, H. Park, B. H. Cho, I. Y. Kim, S. B. Shim, I. Y. Choi, and J. C. Yoo, “Deep-learning framework and
computer assisted fatty infiltration analysis for the supraspinatus muscle in MRI,” vol. 11, no. 1, p. 15065, July, 2021, https:
//doi.org/10.1038/s41598-021-93026-w.
[25] B. A. Polascik, J. Peck, N. Cepeda, S. Lyman, and D. Ling, “Reporting Clinical Significance in Hip Arthroscopy: Where Are
We Now?” vol. 16, pp. 527–533, December, 2020, https://doi.org/10.1007/s11420-020-09759-3.
[26] R. A. Binder, G. F. Fujimori, C. S. Forconi, G. W. Reed, L. S. Silva, P. S. Lakshmi, A. Higgins, L. Cincotta, P. Dutta, M.-
C. Salive, V. Mangolds, O. Anya, J. M. Calvo Calle, T. Nixon, Q. Tang, M. Wessolossky, Y. Wang, D. A. Ritacco, C. S.
Bly, S. Fischinger, C. Atyeo, P. O. Oluoch, B. Odwar, J. A. Bailey, A. Maldonado-Contreras, J. P. Haran, A. G. Schmidt,
L. Cavacini, G. Alter, and A. M. Moormann, “SARS-CoV-2 Serosurveys: How Antigen, Isotype and Threshold Choices Affect
the Outcome,” vol. 227, no. 3, pp. 371–380, Februari, 2023, https://doi.org/10.1093/infdis/jiac431.
[27] M. Franceschini, A. Boffa, E. Pignotti, L. Andriolo, S. Zaffagnini, and G. Filardo, “The Minimal Clinically Important Difference
Changes Greatly Based on the Different Calculation Methods,” vol. 51, no. 4, pp. 1067–1073, March, 2023, https://doi.org/10.
1177/03635465231152484.
[28] J. Bordon, “The Importance of Cycle Threshold Values in the Evaluation of Patients with Persistent Positive PCR for SARSCoV-
2: Case Study and Brief Review,” vol. 4, no. 1, pp. 1–5, 2020, https://doi.org/10.18297/jri/vol4/iss1/54.
[29] J. Bullard, K. Dust, D. Funk, J. E. Strong, D. Alexander, L. Garnett, C. Boodman, A. Bello, A. Hedley, Z. Schiffman, K. Doan,
N. Bastien, Y. Li, P. G. Van Caeseele, and G. Poliquin, “Predicting Infectious Severe Acute Respiratory Syndrome Coronavirus
2 From Diagnostic Samples,” vol. 71, no. 10, pp. 2663–2666, December, 2020, https://doi.org/10.1093/cid/ciaa638.
[30] S. Jayatilake, J. M. Bunker, A. Bhaskar, and M. Miska, “Time–space analysis to evaluate cell-based quality of service in bus
rapid transit station platforms through passenger-specific area,” vol. 13, no. 2, pp. 395–427, June, 2021, https://doi.org/10.1007/
s12469-021-00267-z.
[31] M. F. Bonner and R. A. Epstein, “Object representations in the human brain reflect the co-occurrence statistics of vision and
language,” vol. 12, no. 1, p. 4081, Februari, 2021, https://doi.org/10.1038/s41467-021-24368-2.
[32] C. Qu, M. F. Bonner, N. K. DeWind, and E. M. Brannon, “Contextual coherence increases perceived numerosity independent
of semantic content.” vol. 153, no. 8, pp. 2028–2042, August, 2024, https://doi.org/10.1037/xge0001595.
[33] M. C. Iordan, T. Giallanza, C. T. Ellis, N. M. Beckage, and J. D. Cohen, “Context Matters: Recovering Human Semantic
Structure from Machine Learning Analysis of Large-Scale Text Corpora,” vol. 46, no. 2, p. e13085, Februari, 2022, https:
//doi.org/10.1111/cogs.13085.
[34] B. Cao and J. Liu, “Combining bidirectional long short-term memory and self-attention mechanism for code search,” vol. 35,
no. 10, p. e7662, May, 2023, https://doi.org/10.1002/cpe.7662.
[35] R. Richie and S. Bhatia, “Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison,” vol. 45,
no. 8, p. e13030, August, 2021, https://doi.org/10.1111/cogs.13030.
[36] D. Rose and P. Bex, “The Linguistic Analysis of Scene Semantics: LASS,” vol. 52, no. 6, pp. 2349–2371, December, 2020,
https://doi.org/10.3758/s13428-020-01390-8.
[37] J. C. Yang, “The prediction and analysis of heart disease using XGBoost algorithm,” vol. 41, no. 1, pp. 61–68, Februari, 2024,
https://doi.org/10.54254/2755-2721/41/20230711.
[38] L. Xu, S. Liu, S. Wang, D. Sun, and N. Li, “Word’s Predictability Can Modulate Semantic Preview Effect in High-Constraint
Sentences,” vol. 13, p. 849351, March, 2022, https://doi.org/10.3389/fpsyg.2022.849351.
[39] A. Onan and H. Alhumyani, “Contextual Hypergraph Networks for Enhanced Extractive Summarization: Introducing Multi-
Element Contextual Hypergraph Extractive Summarizer (MCHES),” vol. 14, no. 11, p. 4671, May, 2024, https://doi.org/10.
3390/app14114671.
[40] Y. Zhu, W. Zheng, and H. Tang, “Interactive Dual Attention Network for Text Sentiment Classification,” vol. 2020, pp. 1–11,
March, 2020, https://doi.org/10.1155/2020/8858717.
[41] B. Tang, J. Wang, H. Qiu, J. Yu, Z. Yu, and S. Liu, “Attack Behavior Extraction Based on Heterogeneous Cyberthreat Intelligence
and Graph Convolutional Networks,” vol. 74, no. 1, pp. 235–252, 2023, https://doi.org/10.32604/cmc.2023.029135.
[42] N. H. Hameed, A. M. Alimi, and A. T. Sadiq, “Short Text Semantic Similarity Measurement Approach Based on Semantic
Network,” vol. 19, p. 1581, May, 2022, https://doi.org/10.21123/bsj.2022.7255.
[43] W. Pasisingi, A. Mariana, and D. Husain, “A Semantic Analysis on Maroon 5 Songs,” vol. 2, no. 1, August, 2022, https:
//doi.org/10.30984/jeltis.v2i1.1948.
[44] L. Ding, H. Tang, and L. Bruzzone, “LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote
Sensing Images,” vol. 59, no. 1, pp. 426–435, Januari, 2021, https://doi.org/10.1109/TGRS.2020.2994150.
[45] W. Ma, Y. Wu, F. Cen, and G. Wang, “MDFN: Multi-scale deep feature learning network for object detection,” vol. 100, p.
107149, April, 2020, https://doi.org/10.1016/j.patcog.2019.107149.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Nadindra Dwi Ariyanta, Didik Dwi Prasetya, Ilham Ari Elbaith Zaeni, Tsukasa Hirashima, Reo Wicaksono
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
How to Cite
Similar Articles
- Muhammad Zaki Wiryawan, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Wahyu Styo Pratama, Lalu Ganda Rady Putra, Enhancing Semantic Similarity in Concept Maps Using LargeLanguage Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- Reo Wicaksono, Didik Dwi Prasetya, Ilham Ari Elbaith Zaeni, Nadindra Dwi Ariyanta, Tsukasa Hirashima, Machine Learning for Open-ended Concept Map Proposition Assessment: Impact of Length on Accuracy , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 25 No. 1 (2025)
- Wahyu Styo Pratama, Didik Dwi Prasetya, Triyanna Widyaningtyas, Muhammad Zaki Wiryawan, Lalu Ganda Rady Putra, Tsukasa Hirashima, Performance Evaluation of Artificial Intelligence Models for Classification in Concept Map Quality Assessment , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- F.ti Ayyu Sayyidul Laily, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Revealing Interaction Patterns in Concept Map Construction Using Deep Learning and Machine Learning Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Reni Fatrisna Salsabila, Didik Dwi Prasetya, Triyanna Widyaningtyas, Tsukasa Hirashima, Comparison of Text Representation for Clustering Student Concept Maps , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Siti Ummi Masruroh, Cong Dai Nguyen, Doni Febrianus, Comparative Analysis of TF-IDF and Modern Text Embedding for theClassification of Islamic Ideologies on Indonesian Twitter , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 25 No. 1 (2025)
- Frans Mikael Sinaga, Sio Jurnalis Pipin, Sunaryo Winardi, Karina Mannita Tarigan, Ananda Putra Brahmana, Analyzing Sentiment with Self-Organizing Map and Long Short-Term Memory Algorithms , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 1 (2023)
- Husain Husain, I Putu Hariyadi, Kurniadin Abd Latif, Galih Tri Aditya, Implementation of Port Knocking with Telegram Notifications to Protect Against Scanner Vulnerabilities , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 23 No. 1 (2023)
- Saiful Nur Arif, Muhammad Dahria, Sarjon Defit, Dicky Novriansyah, Ali Ikhwan, Implementation of Single Linked on Machine Learning for Clustering Student Scientific Fields , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 22 No. 1 (2022)
- Anthony Anggrawan, Azhari Azhari, APLIKASI DETEKSI KEMIRIPAN TUGAS PAPER , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 15 No. 2 (2016)
You may also start an advanced similarity search for this article.
Most read articles by the same author(s)
- Wahyu Styo Pratama, Didik Dwi Prasetya, Triyanna Widyaningtyas, Muhammad Zaki Wiryawan, Lalu Ganda Rady Putra, Tsukasa Hirashima, Performance Evaluation of Artificial Intelligence Models for Classification in Concept Map Quality Assessment , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- F.ti Ayyu Sayyidul Laily, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Revealing Interaction Patterns in Concept Map Construction Using Deep Learning and Machine Learning Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Reni Fatrisna Salsabila, Didik Dwi Prasetya, Triyanna Widyaningtyas, Tsukasa Hirashima, Comparison of Text Representation for Clustering Student Concept Maps , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 2 (2025)
- Muhammad Zaki Wiryawan, Didik Dwi Prasetya, Anik Nur Handayani, Tsukasa Hirashima, Wahyu Styo Pratama, Lalu Ganda Rady Putra, Enhancing Semantic Similarity in Concept Maps Using LargeLanguage Models , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 24 No. 3 (2025)
- Reo Wicaksono, Didik Dwi Prasetya, Ilham Ari Elbaith Zaeni, Nadindra Dwi Ariyanta, Tsukasa Hirashima, Machine Learning for Open-ended Concept Map Proposition Assessment: Impact of Length on Accuracy , MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer: Vol. 25 No. 1 (2025)
.png)
