TUTCRIS - Tampereen teknillinen yliopisto

TUTCRIS

Elastic Neural Networks for Classification

Tutkimustuotosvertaisarvioitu

Yksityiskohdat

AlkuperäiskieliEnglanti
OtsikkoProceedings 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2019
KustantajaIEEE
Sivut251-255
Sivumäärä5
ISBN (elektroninen)9781538678848
DOI - pysyväislinkit
TilaJulkaistu - 1 maaliskuuta 2019
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE International Conference on Artificial Intelligence Circuits and Systems - Hsinchu, Taiwan
Kesto: 18 maaliskuuta 201920 maaliskuuta 2019

Conference

ConferenceIEEE International Conference on Artificial Intelligence Circuits and Systems
MaaTaiwan
KaupunkiHsinchu
Ajanjakso18/03/1920/03/19

Tiivistelmä

In this work we propose a framework for improving the performance of any deep neural network that may suffer from vanishing gradients. To address the vanishing gradient issue, we study a framework, where we insert an intermediate output branch after each layer in the computational graph and use the corresponding prediction loss for feeding the gradient to the early layers. The framework-which we name Elastic network-is tested with several well-known networks on CIFAR10 and CIFAR100 datasets, and the experimental results show that the proposed framework improves the accuracy on both shallow networks (e.g., MobileNet) and deep convolutional neural networks (e.g., DenseNet). We also identify the types of networks where the framework does not improve the performance and discuss the reasons. Finally, as a side product, the computational complexity of the resulting networks can be adjusted in an elastic manner by selecting the output branch according to current computational budget.