Avoiding overfiting: new algorithms to improve generalisation in convolutional neural networks

Abstract

Deep Learning has achieved state-of-the-art results in several domains, such as image processing, natural language processing, and audio processing. To accomplish such results, it uses neural networks with several processing layers along with a massive amount of labeled information. One particular family of Deep Learning is the Convolutional Neural Networks (CNNs), which works using convolutional layers derived from the digital signal processing area, being very helpfull to detect relevant features in unstructured data, such as audio and pictures. One way to improve results on CNN is to use regularization algorithms, which aim to make the training process harder but generate models that generalize better for inference when use in applications. The present work contributes in the area of regularization methods for CNNs, proposing more methods for using in different image processing tasks. This thesis presents a collection of works developed by the author during the research period, which were published or submited until present time, presenting: (i) a survey, listing recent regularization works and highlighting the solutions and problems of the area; (ii) a neuron droping method to use in the tensors generated during CNNs training; (iii) a variation of the mentioned method, changing the droping rules, targeting different features of the tensor; and (iv) a label regularization algorithm used in different image processing problems.