Grammar-based Neuroevolution of Fully Convolutional Networks

Abstract

The design of complex and deep neural networks is often performed by identifying and combining building blocks and progressively selecting the most promising combination. Neuroevolution automates this process by employing evolutionary algorithms to guide the search. Within this field, grammar-based evolutionary algorithms have been demonstrated to be powerful tools to describe and thus encode complex neural architectures effectively. In this context, this research proposes a novel grammar-based multi-objective neuroevolutionary approach for generating fully convolutional networks. The proposed method, named Multi-Objective gRammatical Evolution for FUlly convolutional Networks (MOREFUN), includes a new efficient way to encode skip-connections, facilitating the description of complex search spaces and the injection of domain knowledge in the search procedure, the generation of fully convolutional networks upsampling of lower-resolution inputs in multi-input layers, the usage of multi-objective fitness, and the inclusion of data augmentation and optimizer settings in the grammar. The best networks found by the algorithm outperformed those generated by previous grammar-based evolutionary algorithms, achieving 90% accuracy on CIFAR-10 without using transfer learning, ensembles, or test-time data augmentation, while having a relatively small number of parameters.