Classificação de caracteres japoneses cursivos utilizando modelos de aprendizado profundo

Abstract

With recent advances in machine learning, image classification has become essential in various fields of science. In linguistics, tasks such as deciphering, transcribing, and preserving ancient documents are benefited from the application of deep learning models. The study of cursive Japanese characters, for example, involves a type of writing that differs significantly from its modern version, to the extent that only a small portion of Japanese speakers can understand it fluently. Furthermore, due to the way the Japanese syllabary has evolved throughout history, the application of deep learning methods is appropriate for the task of classifying these ancient Japanese writings, given that there are dozens of variants for each character. Thus, this work aimed to develop a classification model capable of predicting cursive Japanese characters in ancient scripts. To achieve this, four traditional classification methods - logistic regression, k-nearest neighbors, support vector machines and boosting - were adjusted, as well as three architectures of a deep learning method - convolutional neural networks - using datasets related to handwritten cursive characters. As a final result, the model that achieved the best performance was a convolutional neural network, with an accuracy of 92.9% in classifying the 49 classes that make up the hiragana syllabary. In addition, a U-Net model was developed to extract characters from new pages of handwritten texts. However, when applying the trained classifier to real data, the accuracy dropped to 69.1%, highlighting the challenges associated with the variability of writing styles and the presence of noise in scanned images.