004.855.5 Neural network methods for solving the problem of credit scoring

Kadiev A. D. (Bauman Moscow State Technical University), Chibisova A. V. (Bauman Moscow State Technical University)

NEURAL NETWORKS, MACHINE LEARNING, RANDOM FOREST, LOGISTIC REGRESSION, GRADIENT BOOSTING


doi: 10.18698/2309-3684-2022-4-8192


The mathematical derivation of the presented neural network model is demonstrated. Reduction of the classification problem to an optimization problem. Produced recon-naissance data analysis, as well as their preprocessing for further use in training classification algorithms. The architectures of neural networks were designed depending on the activation function, the number of hidden layers of the neural network and the number of neurons in the hidden layers. More than ten neural networks were trained to solve the task of credit scoring. The calculation of the learning time of neural networks was made. The solution of the problem using classical machine learning algorithms is presented. It could be seen that the standard deviation of accuracy and ROC AUC for neural networks is greater than that of a random forest. This is due to the fact that we choose the initial weights randomly and calculate the gradients not on the entire sample, but on small parts, which adds some learning error. But these deviations were not only for the worse. In the best situations, according to both metrics, neural networks showed the worst result by a couple of percent. The analysis of results is made. Comparative analysis shows that neural networks have better classification quality than classical machine learning algorithms, and also that neural networks have less training time than classical machine learning algorithms. Graphs and tables displaying the results obtained are presented.


Hastie T., Tibshirani R., Friedman J. The elements of statistical learning: data mining, inference, and prediction. New York, Sptringer, 2009, 745 p.
Marsland S. Machine learning: an algorithmic perspective. Chapman and Hall/CRC, 2009, 406 p.
Haykin S. Neural networks: a comprehensive foundation. Second edition. Ontario, Canada. Prentice Hall, 1999, 823 p.
Goodfellow I., Bengio Y., Courville A. Deep learning. The MIT Press, 2016, 800 p.
Coelho L.P., Richert W. Building machine learning systems with Python. Packt Publishing Ltd, 2013, 290 p.
Vorontsov K.V. Mashinnoe obuchenie: kurs lekcij [Machine learning: a course of lectures] [Electronic resource], 2016. URL: http://www.machinelearning.ru/ (accessed: 15.10.2022)
Bellman R. The Bellman continuum: a collection of the works of Richard E. Bellman. World Scientific Publishing Company, 1987, 892 p.
Bellman R. Adaptive control processes: a guided tour. Princeton University Press, 2015, 276 p.
Powell W.B. Approximate dynamic programming: solving the curses of dimensionality. Wiley, 2011, 656 p.
Marimont R.B. Shapiro M.B. Nearest neighbour searches and the curse of dimensionality. Journal of the Institute of Mathematics and Its Applications, 1979, vol. 24, iss. 1, pp. 59–70.
Radovanovic M., Nanopoulos A., Ivanovic M. Hubs in space: popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 2010, vol. 11, pp. 2487–2531.
Spencer J. Ten lectures on the probabilistic method. Philadelphia, SIAM Publ., 1994, 83 p.
de Grey A.D.N.J. The chromatic number of the plane is at least 5. Cornell University, 2018. DOI: arXiv:1804.02385
Lekun Ya. Kak uchitsya mashina. Revolyuciya v oblasti nejronnyh setej i glubokogo obucheniya [How the machine learns. A revolution in neural networks and deep learning]. Moscow, Intellektual'naya Literatura Publ., 2021, 335 p.
Werbos P.J. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 1990, vol. 78, iss. 10, pp. 1550–1560


Кадиев А.Д., Чибисова А.В. Нейросетевые методы решения задачи кредитного скоринга. Математическое моделирование и численные методы, 2022, № 4, с. 81–92.



Download article

Количество скачиваний: 97