Improved artificial neural network based on hidden layer nodes number adaptive selection and genetic algorithm optimizing parameters
DOI:
https://doi.org/10.14311/NNW.2018.%25xAbstract
The neural network model based on back-propagation (BP) algorithm is a widely used prediction model. However, the nodes number of the first hidden layer, the learning rate and momentum factor are usually determined manually, which affect the network forecast accuracy. Therefore, in this paper, to improve the forecast accuracy, firstly, the nodes number of the first hidden layer is selected adaptively based on minimizing mean square error (MSE). Secondly, improved genetic algorithm (GA) is proposed to train the learning rate and momentum factor dynamically. Thirdly, we construct a new neural network model based on the adaptively selected nodes number of the first hidden layer, the dynamically selected learning rate and momentum factor, which is called HN-GA-BP neural network model. Finally, the proposed neural network model is used to forecast the carbon dioxide levels in China for fifty years. Experimental results demonstrate the effectiveness of the proposed HN-GA-BP neural network model.References
A.F. Sheta, N. Ghatasheh, H. Faris, Forecasting Global Carbon Dioxide Emission Using Auto-Regressive with eXogenous Input and Evolutionary Product Unit Neural Network Models, 6th International Conference on Information and Communication Systems (ICICS), 2015, pp. 182-187.
China Meteorological Administration
<http://www.cma.gov.cn/> (accessed 2017.08.10).
A Askarzadeh, A Rezazadeh, Artificial neural network training using a new efficient optimization algorithm, Applied Soft Computing, 2013, 13 (2) 1206-1213.
A. Prieto, B. Prieto, EM Ortigosa, E. Ros, F. Pelayo, Neural networks: An overview of early research, current frameworks
and new challenges, Neurocomputing, 2016, (214) 242-268.
F. Li, C. Wu, K. Wu, J. Xu, An Improved Back Propagation Neural Network Model and Its Application, Journal of Computers, 9 (8) (2014) 1858-1862.
JP Skon, M Johansson, M Raatikainen, Modelling indoor air carbon dioxide concentration using neural network, World Academy of Science Engineering & Technology, 2012, 14 (15).
V. Bevilacqua, F. Intini, S. Kuhtz, A model of artificial neural network for the analysis of climate change, the 29th Annual International Sympsium on Forecasting, 2008.
C Gallo, F Conto, M Fiore, A neural network model for forecasting co2 emission, AGRIS on-line Papers in Economics and Informatics, 2014, 6 (2).
A. Jamli, P. Ahmadi, M.N.M. Jaafar, Optimization of a novel carbon dioxide cogeneration system using artificial neural network and multi-objective genetic algorithm, Applied Thermal Engineering, 2014, 64 (1-2) 293-306.
S. Yu, YM. Wei, K. Wang, A PSO-GA optimal model to estimate primary energy demand of China, Energy Policy, 2012, 42 (2) 329-340.
Wei Sun, Yanfeng Xu, Using a back propagation neural network based on improved particle swarm optimization to study the influential factors of carbon dioxide emissions in Hebei Province China, Journal of Cleaner Production, 112 (2016) 1282-1291.
M.T. Hagan, H.B. Demuth, M Beale, Neural Network Design, Boston: PWS, 1996.
J. Peralta, G. Gutiérrez, A. Sanchis, ADANN: Automatic Design of Artificial Neural Networks, Proceedings of the 10th annual conference companion on Genetic and evolutionary computation, 2008, pp. 1863-1870.
S. Feng, W. Xiaochuan, Y. Lei, Analysis 30 neural network cases with Matlab, Beijing, Beihang University Press, 2010.
M. Negnevitsky, Artificial Intelligence a Guide to Intelligent Systems Third Edition, China Machine Press, 2013, pp.111-143.
L. Wang, Y. Zeng, T. Chen, Back propagation neural network with adaptive differential evolution algorithm for time series forecasting, Expert Systems with Application, 42 (2015) 855-863.
C. Mi, J. Yang, S. Li, X. Zhang, D. Zhu, in Prediction of accumulated temperature in vegetation period using artificial neural network, Mathematical & Computer Modelling, 51 (2010) 1453-1460.
N.M. Wagarachchi, A.S. Karunananda, Optimization of multi-layer artificial neural networks using delta values of hidden layers, 2013 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (CCMB), 9 (5) (2013) 80-86.
Y. Xiaodong, Selection of initial weights and thresholds based on the Genetic Algorithm with the optimized Back-Propagation neural network, 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), 2015, pp. 173-177.
L. Wang, H. Liu, F. Chen, D. Chen, F. Qishuai, Identification of Flow Regimes Based on Adaptive Learning and Additional Momentum BP Neural Network, 2016 6thInternational Conference on Instrumentation & Measurement, computer, Communication and Control, 2016, pp. 574-578.
S. Masood, M.N. Doja, P. Chandra, Analysis of weight initialization methods for gradient descent with momentum, 2015 International Conference on Soft Computing Techniques and Implementations (ICSCTI), 2015, pp. 131-136.
SJ Narayanan, RB Bhatt, B Perumal, Improving the accuracy of fuzzy decision tree by direct Back Propagation with adaptive learning rate and momentum factor for user location, Procedia Computer Science, 2016, (89) 506-513.
W.S. McCulloch, W.H. Pitts, A logical calculus of the ideas immanent in nervous activity, Bull. Math. Biophys, 5 (1943) 115–133.
R. Hecht-Nielsen, Kolmogorov's Mapping Neural Network Existence Theorem, 1987.
Y. Zhou, Q. Zhu, H. Huang, Prediction of Acute Hypotensive Episode in ICU Using Chebyshev Neural Network, Journal of Software, 8 (8) (2013) 1923-1931.
H.S. Wang, Y.N. Wang, Y.C. Wang, Cost estimation of plastic injection molding parts through integration of PSO and BP neural network, Expert Systems with Applications, 40 (2013) 418-428.
N. Leema, H. Khanna Nehemiah, A. Kannan, Neural network classifier optimization using Differential Evolution with Global Information and Back Propagation algorithm for clinical datasets, Applied Soft Computing, 49 (2016) 834-844.
J.H. Holland, Adaptation in natural and artificial systems, MIT Press, 1992, 6 (2) 126-137.
F. Ahmad, N.A.M. Isa, Z. Hussain, S.N. Sulaiman, A genetic algorithm based multi-objective optimization of an artificial neural network classifier for breast cancer diagnosis, Neural Computing & Applications, 23 (5) (2013) 1427-1435.
P.F. Yan, Some Views on the Research of Multilayer Feedforward Neural Network, Acta Electronica Sinica, 1999, 27 (5) 82-85.
The Website of DATATANG
<http://www.datatang.com/data/45742>
(accessed 2017.08.10).
Published
Issue
Section
License
Only original research papers (not published or not simultaneously submitted to another journal) will be reviewed. Before the article is printed, the authors are required to transfer the copyright of the published article to the publisher. The copyright transfer form can be downloaded here: http://www.nnw.cz/copyright.html or here.
Effective upon acceptance for publication in NNW, copyright (including all rights there under and including the right to authorise dissemination, distribution, photocopying and reproduction in all media, whether separately or as a part of a journal issue or otherwise) in the work and any modifications of it by the authors is hereby transferred throughout the world and for the full term and all extensions and renewals thereof, to CTU as a representative of publishers of NNW.
The following rights are retained by the author(s):
- Patent and trademark rights and rights to any process or procedure described in the work.
- The right to photocopy or make single electronic copies of the work for their own personal use, including for their own classroom use, or for the personal use of colleagues, provided that the copies are not offered for sale and are not distributed ina systematic way outside of their employing institution. Posting of a preprint version of the work on an public server is permitted. Posting of the published work on a secure network (not accessible to the public) within the Author's institution is permitted. However, posting of the published work on an electronic public server can only be done with written permission of CTU as a representative of the publishers of NNW.
- The right, subsequent to publication, to use the work or any part there of free of charge in a printed compilation of works of their own, such as collected writings or lecture notes, in a thesis, or to expand the work into book-length form for publication.
All copies, paper or electronic or other use of the work must include an indication of the copyright ownership and a full citation of the journal source.
Abstracting is permitted with credit to the source. For all other copying, reprint or republication permission write to the Czech Technical University in Prague.
Responsibility for the contents of all the publisher papers and letters rests upon the authors and not upon the editors of the NNW.
