On the training algorithms for Restricted Boltzmann Machine-Based Models

Abstract

Deep learning techniques have been studied extensively in the last years, due to its good results related to essential tasks on a large range of applications, such as speech and face recognition, as well as objects classification. Among the most employed techniques is the Restrict Boltzmann Machines (RBMs), which are energy-based stochastic neural networks composed of two layers of neurons., i.e., visible and hidden, whose objective is to estimate the connection weights between both layers, generally using Markov chains. Recently, the scientific community spent many efforts on sampling methods, since RBMs effectiveness is directly related to the success of the sampling process. Thereby, the present work contributes with RBMs Learning area, as well as its variants DBNs and DBMs. Further, the work covers the application of meta-heuristic methods concerning a proper fine-tune of these techniques. Moreover, the validation of the model is presented in the context of image reconstruction and pattern recognition. In general, the present work presents different approaches to training these techniques, as well as the evaluation of meta-heuristic methods efficiency in training. Finally, this thesis presents a collection of works developed by the author during the study period, which was published/submitted until the present time, concerning: (i) temperature parameter introduction in DBM formulation, (ii) DBM using adaptive temperature, (iii) DBM meta-parameters optimization through meta-heuristic techniques, and (iv) iRBM meta-parameters optimization through meta-heuristic techniques.