EvoDNN: An evolutionary deep neural network with heterogeneous activation functions

Many problems in Computational Biology and Bioinformatics involve classification, such as the classification of cell samples into malignant (cancer) or benign (normal). For such tasks, we propose EvoDNN, an evolutionary deep neural network that employs an evolutionary algorithm to evolve deep heterogeneous feed-forward neural networks. While the majority of current feed-forward neural networks employ user defined homogeneous activation functions, EvoDNN creates heterogeneous multi-layer networks where each neuron's activation function is not statically defined by the user, but dynamically optimized during evolution. The main advantage offered by EvoDNN lies in that the activation functions do not need to be differentiable. This feature gives users a great degree of flexibility over which activation functions EvoDNN can utilize. This thesis demonstrates how EvoDNN can simultaneously optimize each neuron's weight, bias, and activation function, and empirically shows a superior performance compared to feed-forward neural networks trained with backpropagation method, random forest method, and our earlier approach EvoNN which employed a single hidden layer.