EFFICIENT DEEP CNN FOR HANDWRITTEN DIGIT RECOGNITION: A PYTORCH AND SKORCH APPROACH
Abstract
Handwritten digit recognition remains a benchmark problem for evaluating machine learning and deep learning algorithms. In this study, we propose a simple yet effective convolutional neural network (CNN) architecture for multi-class classification of the MNIST dataset, implemented with PyTorch and Skorch. The model employs hyperparameter optimization via grid search to fine-tune the activation functions, kernel sizes, optimizers, and dropout rates. The experimental results demonstrate state-of-the-art performance, achieving 99% accuracy, precision, recall, and F1-score across all 10 digit classes. Comparative analysis against baseline models—including decision trees, SVMs, ANNs, and conventional CNNs—confirms that the proposed model consistently outperforms prior approaches. These findings highlight the effectiveness of lightweight, well-tuned CNN architectures for digit recognition tasks and demonstrate the utility of PyTorch and Skorch as efficient frameworks for model design, training, and deployment
Keywords:
Handwritten Digit Recognition, Deep Learning, Convolutional Neural Networks, PyTorch, Skorch, MNIST, Hyperparameter OptimizationDownloads
Published
DOI:
https://doi.org/10.5281/zenodo.17136040Issue
Section
How to Cite
License
Copyright (c) 2025 Daniel U. Okon
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Ahlawat, S., & Choudhary, A. (2020). Hybrid CNN-SVM Classifier for Handwritten Digit Recognition. Procedia Computer Science, 167, pp.2554–2560. doi:10.1016/j.procs.2020.03.309.
Ammirato, P., & Berg, A. (2019). A mask-RCNN baseline for probabilistic object detection. [online] Available at: https://arxiv.org/pdf/1908.03621.pdf [Accessed December 22, 2022].
Cardoso, Â. and Wichert, A. (2013). Handwritten digit recognition using biologically inspired features. Neurocomputing, 99, pp.575–580. doi:10.1016/j.neucom.2012.07.027.
De Stefano, C., Fontanella, F., Marrocco, C., & Di Freca, A. S. (2014). GA-based feature selection with ANFIS Approach to Breast Cancer Recurrence. (2016). International Journal of Computer Science Issues, 13(1), pp.36–41. doi:10.20943/ijcsi-201602-3641.
DiCarlo, James J., Zoccolan, D., & Rust, Nicole C. (2012). How does the brain solve visual object recognition? Neuron, 73(3), 415–434. https://doi.org/10.1016/j.neuron.2012.01.010.
EDUCBA. (2022). PyTorch SoftMax | Complete Guide on PyTorch Softmax? [online]. Available from: https://www.educba.com/pytorch-softmax/.
Fourie, C. M. (2003). Deep learning? What deep learning? South African Journal of Higher Education, 17(1). https://doi.org/10.4314/sajhe.v17i1.25201.
Guo Y, Liu Y, Oerlemans A, Lao S, Wu, S. and Lew, M.S. (2016). Deep learning for visual understanding: A review. Neurocomputing [online] 187, pp.27–48. doi:10.1016/j.neucom.2015.09.116.
Hanmandlu, M., & Murthy, O. V. R. (2007). Fuzzy model based recognition of handwritten numerals. Pattern Recognition, 40(6), pp.1840–1854. doi:10.1016/j.patcog.2006.08.014.
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. [online]. Available from: https://openaccess.thecvf.com/content_iccv_2015/papers/He_Delving_Deep_into_ICCV_2015_paper.pdf.
Hu, W., Huang, Y., Wei, L., Zhang, F., Li, H., 2015. Deep Convolutional Neural Networks for Hyperspectral Image Classification. Journal of Sensors, [online] 2015, p.e258619. doi:10.1155/2015/258619.
Jabreel, M., & Moreno, A. (2019). A Deep Learning-Based Approach for Multi-Label Emotion Classification in Tweets. Appl Sci, 9(6), 1123. doi:10.3390/app9061123.
Jiang, L., & Zhang, Z. (2021). Research on Image Classification Algorithm Based on Pytorch. Journal of Physics: Conference Series, 2010(1), p.012009. doi:10.1088/1742-6596/2010/1/012009.
Kadam, S. S., Adamuthe, A. C., & Patil, A. B. (2020). CNN model for image classification on MNIST and fashion-MNIST dataset. Journal of Scientific Research, 64(02), 374–384. https://doi.org/10.37398/jsr.2020.640251.
Kaziha, O., & Bonny, T. (2019). A comparison of quantized convolutional and LSTM recurrent neural network models using MNIST. [Online] IEEE Xplore. https://doi.org/10.1109/ICECTA48151.2019.8959793.
Lauer, F., Suen, C. Y. and Bloch, G. (2007). A trainable feature extractor for handwritten digit recognition. Pattern Recognition, 40(6), pp.1816–1824. doi:10.1016/j.patcog.2006.10.011.
LeCun, L., Henderson, J., Le Cun, Y., Denker, J., Henderson, D., Howard, R., Hubbard, W., & Jackel, L. (2022). Handwritten Digit Recognition with a Back-Propagation Network. Journal of Computer Science. [Online]. Available from: https://proceedings.neurips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf.
Lejeune, E. (2020). Mechanical MNIST: A benchmark dataset for mechanical metamodels. Extreme Mechanics Letters, 36, p.100659. doi:10.1016/j.eml.2020.100659.
Liu, C. L., Nakashima, K., Sako, H., & Fujisawa, H. (2003). Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern Recognit [online] 36(10), pp.2271–2285. Doi: 10.1016/s0031-3203(03)00085-2.
Liu, Y. (2020). 3D Image Segmentation of MRI Prostate Based on a Pytorch Implementation of V-Net. Journal of Physics: Conference Series, 1549, 042074. https://doi.org/10.1088/1742-6596/1549/4/042074.
López, F. (2020). SKORCH: PyTorch Models Trained with a Scikit-Learn Wrapper. [Online] Medium. Available from: https://towardsdatascience.com/skorch-pytorch-models-trained-with-a-scikit-learn-wrapper-62b9a154623e
Mohapatra, R., Majhi, B., & Kumar, S. (2015). Classification performance analysis of MNIST dataset utilizing a multi-resolution technique. [Online]. Available at: http://dspace.nitrkl.ac.in:8080/dspace/bitstream/2080/2403/1/Classification_Mohapatra_2015.pdf [Accessed 30 Dec. 2022].
Niu, X.-X. And Suen, C.Y. (2012). A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recognit 45:1318–1325. https://doi.org/10.1016/j.patcog.2011.09.021.
Overview of classification methods in Python with Scikit-Learn", Stack Abuse, 2022. [Online]. Available: https://stackabuse.com/overview-of-classification-methods-in-python-with-scikit-learn/.
Pourjavan, S. (2019). Definitions: machine learning, deep learning and AI understanding. Acta Ophthalmol. 97, S263. https://doi.org/10.1111/j.1755-3768.2019.8214.
Priyansh, P., Ritu, G., Mazhar, K., & Sajid, I. (2020). Multi-digit number classification using MNIST and ANN. International Journal of Engineering Research and, V9 (05). Doi: 10.17577/ijertv9is050330.
Ragab, M., Alshehri, S., Alhakamy, N. A., Mansour, R. F., & Koundal, D. (2022). Multiclass Classification of Chest X-Ray Images for the Prediction of COVID-19 Using Capsule Network. Computational Intelligence and Neuroscience, 2022, pp.1–8. doi:10.1155/2022/6185013.
Rothman, D. (2021). Transformers for natural language processing: Build innovative deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, RoBERTa, and more. XXXXXXXXXXXX. [online] Google Books. Packt Publishing Ltd. Available from: https://books.google.com.ng/books?hl=en&lr=&id=Cr0YEAAAQBAJ&oi=fnd&pg=PP1&dq=++Rothman [Accessed 27 Dec. 2022].
Savas, B., & Eldén, L. (2007). http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A18008. [Online] Available from: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A18008&dswid=5745 [Accessed 20 Dec. 2022].
Shamsuddin, M. R., Abdul-Rahman, S., & Mohamed, A. (2018). Exploratory Analysis of MNIST Handwritten Digit for Machine Learning Modeling. Communications in Computer and Information Science, pp.134–145. Doi: 10.1007/978-981-13-3441-2_11.
Sikka, B. (2021). Elements of deep learning for computer vision: explore deep neural network architectures, PyTorch, object detection algorithms, and computer vision applications for Python coding. [Online] Google Books. BPB Publications. Available from: https://books.google.com.ng/books?hl=en&lr=&id=n8U0EAAAQBAJ&oi=fnd&pg=PT19&dq=Sikka [Accessed 24 Dec. 2022].
Tang, Y. (2013). Deep learning using linear support vector machines. [Online]. Available at: https://arxiv.org/pdf/1306.0239.pdf.
Wen, Y. (2012). An improved discriminative common vectors and support vector machine based face recognition approach. Expert Syst Appl, 39(4), 4628–4632. doi:10.1016/j.eswa.2011.09.119.
Xue, D.-X., Zhang, R., Feng, H., & Wang, Y.-L. (2016). CNN-SVM for microvascular morphological type recognition with data augmentation. Journal of Medical and Biological Engineering, 36(6), pp.755–764. doi:10.1007/s40846-016-0182-4.
Alqahtani, A., & Wang, H. (2024). Advances in Deep Convolutional Networks for Visual Recognition: A Survey IEEE Access, 12, 12501-12520. doi: 10.1109/ACCESS.2024.3352104
Zhang, Y., Sun, Y., & Lin, X. (2023). Scalable DL for image recognition in big data environments Future Generation Computer Systems, 144, 85-97. https://doi.org/10.1016/j.future.2023.01.005
Doshi, K., Patel, D., & Shah, H. (2022). Lightweight Convolutional Neural Networks for Handwritten Digit Recognition Procedia Comput Sci, 198, 12-20. https://doi.org/10.1016/j.procs.2021.12.003
Rahman, M., & Hossain, S. (2023). Hyperparameter optimization of convolutional neural network models for handwritten digit classification. Applied Intelligence, 53(7), 8451-8463. doi: 10.1007/s10489-022-04125-6
Han, J., & Zhang, W. (2022). PyTorch-based deep learning frameworks for computer vision applications. Journal of Imaging, 8(9), 238. doi: 10.3390/jimaging8090238
Khan M, Akhtar N, Rehman S. 2021. Hybrid Convolutional Neural Network architectures for image classification: A case study on MNIST and CIFAR datasets International Journal of Machine Learning and Cybernetics, 12(11), 3313-3327. DOI: 10.1007/s13042-021-01342-7
Singh, A., Sharma, R., & Kumar, V. (2022). Robust lightweight convolutional neural networks for handwritten digit recognition under adversarial attacks. Neural Computing and Applications, 34(21), 19157-19172. DOI: 10.1007/s00521-022-07450-9
