AUTOMATED SPEECH RECOGNITION TRANSCRIPTION AND SIGN LANGUAGE CHARACTER ANIMATION ON LEARNING MEDIA

Authors

DOI:

https://doi.org/10.18623/rvd.v22.n7.4211

Keywords:

Learning Media, ASR Transcription, Sign Language Animation

Abstract

Inclusive education for deaf students requires a technology approach to address communication and comprehension challenges. This study aims to develop innovative learning media that integrate real-time ASR (Automated Speech Recognition) transcription technology and animation of sign language characters to improve accessibility and comprehension of materials for deaf students. This learning media receives input from live voice, voice from learning videos, and text inputted by teachers. Using the Google Cloud API-based ASR transcription module, voice and text are converted into written text, broken down into vocabulary for sign language animation search. The search is carried out using an interpolation algorithm in the sign language animation asset database, allowing the display of animations relevant to the transcribed vocabulary. The development process follows the ADDIE instructional design model, starting with needs analysis and ending with implementation and evaluation. The analysis stage includes data collection through teacher interviews, classroom observations, and curriculum reviews. The media design is designed to meet the specific needs of deaf students, while development and implementation focus on technology integration and effective material delivery. Evaluation is carried out to assess the effectiveness of the media in improving student understanding and participation. The study's results showed that this learning media can improve deaf students' understanding of the material and increase their involvement in the learning process. ASR technology and sign language animation contribute significantly to making learning materials more accessible and understandable, supporting the goals of inclusive education.

References

Aditya, M., Wardana, W., Febriana, N., Karina, Y. K., Mulyono, S., Aditya, M., Wardana, W., & Karina, Y. K. (2022). Pengembangan Media Pembelajaran Pull Out Photo Box Sebagai Upaya Peningkatan Pemerolehan dan Pembelajaran Bahasa Bagi Anak Berkebutuhan Khusus (ABK) pada Sekolah Inklusi Tingkat Dasar. Jurnal Improvement, 9(1), 42–54.

Agung, S., Khoirunisa, A. N., & Suryaningsih, S. (2022). Tantangan Guru Sekolah Luar Biasa Pada Pembelajaran Ipa Di Abad 21. Alotrop, 6(1), 43–52. https://doi.org/10.33369/alo.v6i1.21089

Aji, M. P., & Aeman, U. G. (2023a). Aplikasi Speech Recognition untuk Audio Forensik. Jurnal Media Pratama, 17(1), 61–73.

Aji, M. P., & Aeman, U. G. (2023b). Aplikasi speech recognition untuk audio forensik. Jurnal Media Pratama, 17(1), 61–73.

Arakawa, R., Yakura, H., & Goto, M. (2022). BeParrot: Efficient Interface for Transcribing Unclear Speech via Respeaking. International Conference on Intelligent User Interfaces, Proceedings IUI, 832–840. https://doi.org/10.1145/3490099.3511164

Azzahra, A., Mawarsari, V. D., & Suprayitno, I. J. (2025). Inovasi pendidikan inklusif: Analisis kebutuhan media pembelajaran interaktif berbasis teknologi untuk siswa tunarungu. Journal Genta Mulia, 16(1), 75–87.

Branch, R. M. (2009). Approach, Instructional Design: The ADDIE. In Department of Educational Psychology and Instructional Technology University of Georgia (Vol. 53, Issue 9).

Carrier, M. (2017). Automated speech recognition in language learning: Potential models, benefits and impact. Training, Language and Culture, 1(1), 46–61. https://doi.org/10.29366/2017tlc.1.1.3

Dewi, N. M., & Qamariah, Z. (2023). The function and role of needs analysis in English learning curriculum. Jurnal Ilmiah Multidisiplin, 2(2), 134–140. https://doi.org/https://doi.org/10.56127/jukim.v2 i02.708

Ekayati, R., Manurung, I. D., & Yenni, E. (2020). Need analysis of ESP for non-English study program. Language Literacy: Journal of Linguistics, Literature and Language Teaching, 4(2), 322–332. https://doi.org/10.30743/ll.v4i2.3152

Elez, P. de la P., & Martín, V. R. (2022). Deaf students in the classroom, inclusive bilingual education: systematic review. International Humanities Review, 11. https://doi.org/10.37467/revhuman.v11.4018

Gunawan, D. (2016). Modul Guru Pembelajar SLB Tunarungu Kelompok Kompetensi A. Pppptk Tk Dan Plb Bandung.

Hornby, G. (2011). Inclusive education for children with special educational needs: A critique. In International Journal of Disability, Development and Education (Vol. 58, Issue 3, pp. 321–329). https://doi.org/10.1080/1034912X.2011.598678

Khairuddin. (2020). Pendidikan Inklusif Di Lembaga Pendidikan. Jurnal Tazkiya, Vol. 9(No. 1), 82–104.

Mabruroh, A. F., & Widyastono, H. (2022). Exploration of Learning Implementation for Deaf Students During the Covid-19 Pandemic. 11(2), 265–273.

Mansur, H., Utama, A. H., Yasin, M. H. M., Sari, N. P., Jamaludin, K. A., & Pinandhita, F. (2023). Development of Inclusive Education Learning Design in the Era of Society 5.0. Social Sciences, 496. https://doi.org/10.4337/9781788974912.S.41

Maria, U., Rusilowati, A., Hardyanto, W., Tengah, C., & Tengah, J. (2019). Interactive multimedia development in the learning process of Indonesian culture introduction theme for 5-6 year old children. Journal of Primary Education, 8(3), 344–353.

Nasri, A. (2020). Konversi Suara Ucapan Bahasa Indonesia ke Sistem Bahasa Isyarat Indonesia (SIBI). AINET: Jurnal Informatika, 2(2), 7–13.

Nasution, F., Anggraini, L. Y., & Putri, K. (2022). Pengertian pendidikan, sistem pendidikan sekolah luar biasa, dan jenis-jenis sekolah luar biasa. In Jurnal Edukasi Nonformal. core.ac.uk.

Pradipta, R. F., Efendi, M., Huda, A., & ... (2021). Comparative study: Use of ICT media in learning for deaf students during the covid-19 pandemic in Malaysia and Indonesia. … on Education and …, 601(Icet), 182–188.

Puspita, S., & Nurhalim, A. D. (2021). Importance of training needs analysis for human resources development in organizations. Jurnal Ekonomi Manajemen Dan Bisnis, 22(2), 151–160.

Ramadania, F., Kisyani, K., & Mintowati, M. (2020). Creation of Audiovisual Media for Children with Special Needs in Indonesian Language Learning. 129–139.

Ratriyana, I., & Nuswantoro, R. (2022). Developing Integrated Multimedia Communication for Deaf Students in Higher Education. Journal of Disability Studies, 09.

Saadah, N., & Harsiwi, N. E. (2024). Analisis Fasilitas Dan Ekstrakurikuler Siswa Di SLB Negeri Keleyan. 2(2), 213–224.

Sudarma, I. K., Ilia, W., & Sukmana, Y. (2021). Improving children’s cognitive ability through information processing theory-based digital content. International Journal of Elementary Education, 6(1), 118–126. https://doi.org/https://dx.doi.org/10.23887/ijee.v6i1v

Verlinden, M., Netherlands, T., Zwitserlood, I., & Frowein, H. (2005). Multimedia with Animated Sign Language for Deaf Learners. January 2005, 4759–4764.

Widiartha, K. K., Agustini, K., Tegeh, I. M., & Warpala, I. W. S. (2024). Real time automated speech recognition transcription and sign language character animation on learning media. Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), 13(3). https://doi.org/10.23887/janapati.v13i3.85065

Wilyanti, L. S., Larlen, L., & Wulandari, S. (2022). Transkripsi Sastra Lisan Melayu Jambi sebagai Alternatif Bahan Ajar Sastra di Perguruan Tinggi. Jurnal Ilmiah Universitas Batanghari Jambi, 22(1), 247. https://doi.org/10.33087/jiubj.v22i1.1889

Zaki, H. M. (2020). The implementation of online-based learning method in Madrasah Ibtidaiyah Negeri 2 Mataram. Jurnal Tatsqif, 18(1), 36–59. https://doi.org/10.20414/jtq.v18i1.2574

Zhang, J., Wu, J., Qiu, Y., Song, A., Li, W., Li, X., & Liu, Y. (2023a). Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review. Computers in Biology and Medicine, 153(January), 106517. https://doi.org/10.1016/j.compbiomed.2022.106517

Zhang, J., Wu, J., Qiu, Y., Song, A., Li, W., Li, X., & Liu, Y. (2023b). Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review. Computers in Biology and Medicine, 153(January, 106517). https://doi.org/https://doi.org/10.1016/j.compbiomed.2022.106517

Downloads

Published

2025-12-30

How to Cite

Widiartha, K. K., Agustini, K., Tegeh, I. M., & Warpala, I. W. S. (2025). AUTOMATED SPEECH RECOGNITION TRANSCRIPTION AND SIGN LANGUAGE CHARACTER ANIMATION ON LEARNING MEDIA. Veredas Do Direito, 22, e224211. https://doi.org/10.18623/rvd.v22.n7.4211

Issue

Vol. 22 (2025): Veredas do Direito, v. 22, 2025

Section

Articles

License

I (we) submit this article which is original and unpublished, of my (our) own authorship, to the evaluation of the Veredas do Direito Journal, and agree that the related copyrights will become exclusive property of the Journal, being prohibited any partial or total copy in any other part or other printed or online communication vehicle dissociated from the Veredas do Direito Journal, without the necessary and prior authorization that should be requested in writing to Editor in Chief. I (we) also declare that there is no conflict of interest between the articles theme, the author (s) and enterprises, institutions or individuals.

I (we) recognize that the Veredas do Direito Journal is licensed under a CREATIVE COMMONS LICENSE.

 

Licença Creative Commons Attribution 3.0