Offline handwriting recognition using deep learning with emphasis on data augmentation effects

We have proposed a deep learning model leveraging train and test time data augmentation approaches for the problem of offline handwriting recognition. We made a comprehensive analysis using our CNN-BiLSTM network to provide evaluation results of each component of the network for the IAM dataset, which is the most commonly used handwriting dataset. We experimented with the deep learning network architecture; evaluated the effects of train time data augmentation and pretraining the network with synthetic handwriting images to alleviate the data sparseness problem. We proposed two different test time augmentation methods as post-processing approaches to obtain the correct transcription of the handwriting image out of the partly correct predicted transcriptions. While the test time augmentation increases the time and computational complexity, it reduced the error rate by 2.5% points and thus could be preferred for batch processes when there are no real-time recognition requirements. Furthermore, we attempt the initial steps of offline handwriting recognition for the Turkish language. To this end, we crafted the first, line-level Turkish handwriting dataset, consisting of more than 2600 line images collected from 73 different people. We applied our deep learning method to this dataset to provide baseline results for future studies; besides, we explored approaches including transfer learning from the IAM to the Turkish dataset and joint training with both datasets. Our contributions include an extensive error analysis for both datasets, revealing better insights into methods and datasets. As part of this effort, we provide our evaluation criteria clearly and completely along with our proposed model to encourage scientific reproducibility.