TY - GEN
T1 - Step Size Control in Evolutionary Algorithms for Neural Architecture Search
AU - Nieber, Christian
AU - Mota Dias, Douglas
AU - Naredo Garcia, Enrique
AU - Ryan, Conor
N1 - Publisher Copyright:
© 2024 by SCITEPRESS – Science and Technology Publications, Lda.
PY - 2024
Y1 - 2024
N2 - This work examines how evolutionary Neural Architecture Search (NAS) algorithms can be improved by controlling the step size of the mutation of numerical parameters. The proposed NAS algorithms are based on F-DENSER, a variation of Dynamic Structured Grammatical Evolution (DSGE). Overall, a (1+5) Evolutionary Strategy is used. Two methods of controlling the step size of mutations of numeric values are compared to Random Search and F-DENSER: Decay of the step size over time and adaptive step size for mutations. The search for lightweight, LeNet-like CNN architectures for MNIST classification is used as a benchmark, optimizing for both accuracy and small architectures. An architecture is described by about 30 evolvable parameters. Experiments show that with step size control, convergence is faster, better performing neural architectures are found on average, and with lower variance. The smallest architecture found during the experiments reached an accuracy of 98.8% on MNIST with only 5,450 free parameters, compared to the 62,158 parameters of LeNet-5.
AB - This work examines how evolutionary Neural Architecture Search (NAS) algorithms can be improved by controlling the step size of the mutation of numerical parameters. The proposed NAS algorithms are based on F-DENSER, a variation of Dynamic Structured Grammatical Evolution (DSGE). Overall, a (1+5) Evolutionary Strategy is used. Two methods of controlling the step size of mutations of numeric values are compared to Random Search and F-DENSER: Decay of the step size over time and adaptive step size for mutations. The search for lightweight, LeNet-like CNN architectures for MNIST classification is used as a benchmark, optimizing for both accuracy and small architectures. An architecture is described by about 30 evolvable parameters. Experiments show that with step size control, convergence is faster, better performing neural architectures are found on average, and with lower variance. The smallest architecture found during the experiments reached an accuracy of 98.8% on MNIST with only 5,450 free parameters, compared to the 62,158 parameters of LeNet-5.
KW - Evolutionary Algorithms
KW - LeNet
KW - Lightweight CNNs
KW - MNIST
KW - Neural Architecture Search
KW - Step Size Control
UR - http://www.scopus.com/inward/record.url?scp=85211430453&partnerID=8YFLogxK
U2 - 10.5220/0013013800003837
DO - 10.5220/0013013800003837
M3 - Conference contribution
AN - SCOPUS:85211430453
SN - 9789897587214
T3 - International Joint Conference on Computational Intelligence
SP - 288
EP - 295
BT - Proceedings of the 16th International Joint Conference on Computational Intelligence, IJCCI 2024
A2 - Marcelloni, Francesco
A2 - Madani, Kurosh
A2 - van Stein, Niki
A2 - Joaquim, Joaquim
PB - Science and Technology Publications, Lda
T2 - 16th International Joint Conference on Computational Intelligence, IJCCI 2024
Y2 - 20 November 2024 through 22 November 2024
ER -