TUTCRIS - Tampereen teknillinen yliopisto

TUTCRIS

Simultaneously Learning Architectures and Features of Deep Neural Networks

Tutkimustuotosvertaisarvioitu

Yksityiskohdat

AlkuperäiskieliEnglanti
OtsikkoArtificial Neural Networks and Machine Learning – ICANN 2019
AlaotsikkoDeep Learning - 28th International Conference on Artificial Neural Networks, Proceedings
ToimittajatIgor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková
KustantajaSpringer Verlag
Sivut275-287
Sivumäärä13
ISBN (painettu)9783030304836
DOI - pysyväislinkit
TilaJulkaistu - 2019
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaInternational Conference on Artificial Neural Networks - Munich, Saksa
Kesto: 17 syyskuuta 201919 syyskuuta 2019

Julkaisusarja

NimiLecture Notes in Computer Science
Vuosikerta11728
ISSN (painettu)0302-9743
ISSN (elektroninen)1611-3349

Conference

ConferenceInternational Conference on Artificial Neural Networks
MaaSaksa
KaupunkiMunich
Ajanjakso17/09/1919/09/19

Tiivistelmä

This paper presents a novel method which simultaneously learns the number of filters and network features repeatedly over multiple epochs. We propose a novel pruning loss to explicitly enforces the optimizer to focus on promising candidate filters while suppressing contributions of less relevant ones. In the meanwhile, we further propose to enforce the diversities between filters and this diversity-based regularization term improves the trade-off between model sizes and accuracies. It turns out the interplay between architecture and feature optimizations improves the final compressed models, and the proposed method is compared favorably to existing methods, in terms of both models sizes and accuracies for a wide range of applications including image classification, image compression and audio classification.

Julkaisufoorumi-taso