Compressing Neural Networks with Two-Layer Decoupling

Joppe De Jonghe; Konstantin Usevich; Philippe Dreesen; Mariya Ishteva

doi:10.1109/CAMSAP58249.2023.10403509

Compressing Neural Networks with Two-Layer Decoupling

Joppe De Jonghe^*
, Konstantin Usevich
, Philippe Dreesen
, Mariya Ishteva

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

40 Downloads (Pure)

Abstract

The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multilayer decoupling problem. As a first step towards general multilayer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.

Original language	English
Title of host publication	2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023
Publisher	IEEE
Pages	226-230
Number of pages	5
ISBN (Electronic)	9798350344523
DOIs	https://doi.org/10.1109/CAMSAP58249.2023.10403509
Publication status	Published - 2023
Event	9th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing - Herradura, Costa Rica Duration: 10 Dec 2023 → 13 Dec 2023 Conference number: 9 https://camsap23.ig.umons.ac.be/

Workshop

Workshop	9th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing
Abbreviated title	CAMSAP 2023
Country/Territory	Costa Rica
City	Herradura
Period	10/12/23 → 13/12/23
Internet address	https://camsap23.ig.umons.ac.be/

Keywords

compression
decoupling
neural network
tensor
tensor decomposition

Access to Document

10.1109/CAMSAP58249.2023.10403509

Full TextFinal published version, 1.36 MBLicence: Taverne

Cite this

@inproceedings{ac0d41b477fa417ba3ac1b33b379c138,

title = "Compressing Neural Networks with Two-Layer Decoupling",

abstract = "The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multilayer decoupling problem. As a first step towards general multilayer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.",

keywords = "compression, decoupling, neural network, tensor, tensor decomposition",

author = "\{De Jonghe\}, Joppe and Konstantin Usevich and Philippe Dreesen and Mariya Ishteva",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 9th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023 ; Conference date: 10-12-2023 Through 13-12-2023",

year = "2023",

doi = "10.1109/CAMSAP58249.2023.10403509",

language = "English",

pages = "226--230",

booktitle = "2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023",

publisher = "IEEE",

address = "United States",

url = "https://camsap23.ig.umons.ac.be/",

}

De Jonghe, J, Usevich, K, Dreesen, P & Ishteva, M 2023, Compressing Neural Networks with Two-Layer Decoupling. in 2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023. IEEE, pp. 226-230, 9th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, Herradura, Costa Rica, 10/12/23. https://doi.org/10.1109/CAMSAP58249.2023.10403509

Compressing Neural Networks with Two-Layer Decoupling. / De Jonghe, Joppe; Usevich, Konstantin; Dreesen, Philippe et al.
2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023. IEEE, 2023. p. 226-230.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - Compressing Neural Networks with Two-Layer Decoupling

AU - De Jonghe, Joppe

AU - Usevich, Konstantin

AU - Dreesen, Philippe

AU - Ishteva, Mariya

N1 - Conference code: 9

PY - 2023

Y1 - 2023

N2 - The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multilayer decoupling problem. As a first step towards general multilayer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.

AB - The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multilayer decoupling problem. As a first step towards general multilayer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.

KW - compression

KW - decoupling

KW - neural network

KW - tensor

KW - tensor decomposition

U2 - 10.1109/CAMSAP58249.2023.10403509

DO - 10.1109/CAMSAP58249.2023.10403509

M3 - Conference article in proceeding

SP - 226

EP - 230

BT - 2023 IEEE 9th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2023

PB - IEEE

T2 - 9th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing

Y2 - 10 December 2023 through 13 December 2023

ER -

Compressing Neural Networks with Two-Layer Decoupling

Abstract

Workshop

Keywords

Access to Document

Fingerprint

Cite this