Data and Documentation
Open Data Policy
FAQ
EN
DE
FR
Suchbegriff
Advanced search
Publication
Back to overview
Information Bottleneck Classification in Extremely Distributed Systems
Type of publication
Peer-reviewed
Publikationsform
Original article (peer-reviewed)
Author
Ullmann Denis, Rezaeifar Shideh, Taran Olga, Holotyak Taras, Panos Brandon, Voloshynovskiy Slava,
Project
Machine Learning based Analytics for Big Data in Astronomy
Show all
Original article (peer-reviewed)
Journal
Entropy
Volume (Issue)
22(11)
Page(s)
1237 - 1237
Title of proceedings
Entropy
DOI
10.3390/e22111237
Open Access
URL
http://doi.org/10.3390/e22111237
Type of Open Access
Publisher (Gold Open Access)
Abstract
We present a new decentralized classification system based on a distributed architecture. This system consists of distributed nodes, each possessing their own datasets and computing modules, along with a centralized server, which provides probes to classification and aggregates the responses of nodes for a final decision. Each node, with access to its own training dataset of a given class, is trained based on an auto-encoder system consisting of a fixed data-independent encoder, a pre-trained quantizer and a class-dependent decoder. Hence, these auto-encoders are highly dependent on the class probability distribution for which the reconstruction distortion is minimized. Alternatively, when an encoding–quantizing–decoding node observes data from different distributions, unseen at training, there is a mismatch, and such a decoding is not optimal, leading to a significant increase of the reconstruction distortion. The final classification is performed at the centralized classifier that votes for the class with the minimum reconstruction distortion. In addition to the system applicability for applications facing big-data communication problems and or requiring private classification, the above distributed scheme creates a theoretical bridge to the information bottleneck principle. The proposed system demonstrates a very promising performance on basic datasets such as MNIST and FasionMNIST.
-