Simultaneously Learning Architectures and Features of Deep Neural Networks
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review
Details
| Original language | English |
|---|---|
| Title of host publication | Artificial Neural Networks and Machine Learning – ICANN 2019 |
| Subtitle of host publication | Deep Learning - 28th International Conference on Artificial Neural Networks, Proceedings |
| Editors | Igor V. Tetko, Pavel Karpov, Fabian Theis, Vera Kurková |
| Publisher | Springer Verlag |
| Pages | 275-287 |
| Number of pages | 13 |
| ISBN (Print) | 9783030304836 |
| DOIs | |
| Publication status | Published - 2019 |
| Publication type | A4 Article in a conference publication |
| Event | International Conference on Artificial Neural Networks - Munich, Germany Duration: 17 Sep 2019 → 19 Sep 2019 |
Publication series
| Name | Lecture Notes in Computer Science |
|---|---|
| Volume | 11728 |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Conference
| Conference | International Conference on Artificial Neural Networks |
|---|---|
| Country | Germany |
| City | Munich |
| Period | 17/09/19 → 19/09/19 |
Abstract
This paper presents a novel method which simultaneously learns the number of filters and network features repeatedly over multiple epochs. We propose a novel pruning loss to explicitly enforces the optimizer to focus on promising candidate filters while suppressing contributions of less relevant ones. In the meanwhile, we further propose to enforce the diversities between filters and this diversity-based regularization term improves the trade-off between model sizes and accuracies. It turns out the interplay between architecture and feature optimizations improves the final compressed models, and the proposed method is compared favorably to existing methods, in terms of both models sizes and accuracies for a wide range of applications including image classification, image compression and audio classification.