Summary
The architecture and working of the Artificial Neural Networks are an inspiration from the human brain. The brain due to its highly parallel nature and immense computational powers still remains the motivation for researchers. A single system-single processor approach is a highly unlikely way to model a neural network for large computational needs. Many approaches have been proposed that adopt a parallel implementation of ANNs. These methods do not consider the difference in processing powers of the constituting units and hence workload distribution among the nodes is not optimal. Human brain not always has equal processing power among the neurons. A person having disability in some part of brain may be able to perform every task with reduced capabilities. Disabilities weaken the processing of some parts. This inspires us to make a self-adaptive system of ANN that would optimally distribute computation among the nodes. The self-adaptive nature of the algorithm makes it possible for the algorithm to taper dynamic changes in node performance. We used data, node and layer partitioning in a hierarchical manner in order to evolve the most optimal architecture comprising of the best features of these partitioning techniques. The adaptive hierarchical architecture enables performance optimisation in whatever condition and problem the algorithm is used. The system was implemented and tested on 20 systems working in parallel. Besides, the computational speed-up, the algorithm was able to monitor changes in performance and adapt accordingly.
See the full content of this document
Extract
Self-Adaptive Parallel Processing Neural Networks with Irregular Nodal Processing Powers Using Hierarchical Partitioning
(ProQuest: ... denotes formulae omitted.)
1. IntroductionArtificial Neural Networks have been applied in numerous domains and problems such as speech recognition, face recognition, character recognition, financial analysis, bioinformatics, credit analysis etc. [16, 2, 34, 47]. Their application over the years has resulted in an increase in the volume and dimensionality of data that require a lot of time to train [30] . The neural networks are inspired from the human brain. The human brain [31] is made up of a number of processing units or neurons working in parallel.With the rise of multi-processing and grid-computing, more and more systems are now being made based on parallel algorithms. Various models of parallel implementation of Back Propagation Algorithm have thus been proposed. The parallelism introduced in neural networks improves its performance especially during the training stage [8, 15, 17, 18, 21]. Parallel neural networks implement the neural networks in parallel by various kinds of partitioning of data set, nodes or layers called as data set partitioning, node partitioning and layer partitioning respectively [3, 5, 6, 42, 44, 48].In this paper we introduce the concept of hierarchical partitioning. Every partitioning technique gives an optimal performance under some conditions. Here we apply all partitioning techniques one after the other in a hierarchical manner. First data set partitioning is applied to distribute test cases among node sets. Then layer partitioning is applied in half of the node sets and node partitioning in the other half. The actual distribution of computation is done in an adaptive manner at runtime. This makes it possible for the most optimal architecture of the system to evolve. This further ensures an optimal distribution of computation among nodes such that no node sits idle for long [25, 34, 36, 47]. This is motivated from the adaptive...See the full content of this document
Sponsored links
