Agenda
Signal Processing Seminar
- Thursday, 29 February 2024
- 11:00-12:00
- HB 17.140
Cauchy-Schwarz Divergence Information Bottleneck for Regression
Shujian YuDepartment of Artificial Intelligence, VU, Amsterdam
The information bottleneck (IB) approach is popular to improve the generalization and robustness of deep neural networks. Essentially, it aims to find a minimum sufficient representation t from input variable x that is relevant for predicting desirable response variable y, by striking a trade-off between I(x;t) and I(y;t), where I refers to the mutual information (MI). However, optimizing IB remains a difficult problem. In this talk, we study the IB principle for the regression problem and develop a new way to parameterize IB with deep neural networks, by leveraging the favorable properties of the Cauchy-Schwarz (CS) divergence. By doing so, we move away from the mean squared error (MSE) loss-based regression and ease estimation of MI terms by avoiding variational approximations or distributional assumptions. We investigate the improved generalization ability of our proposed CS-IB and demonstrate strong adversarial robustness guarantee. We observe that the solutions discovered by CS-IB always achieve the best trade-off between prediction accuracy and compression ratio. We additionally extend CS-IB to structured data such as graphs, and demonstrate its effectiveness to predict the age of patients based on their brain functional MRI (fMRI) data with a graph neural network.
Additional information ...Agenda
- Wed, 18 Sep 2024
- 17:30
- Aula Senaatszaal
PhD Thesis Defence
Wietse Bouwmeester
Automotive Polarimetric Radar for Enhanced Road Surface Classification & Vulnerable Road User Identification
- Wed, 2 Oct 2024
- 17:30
- Aula Senaatszaal
PhD Thesis Defence
Sen Yuan
3D Motion-Based High-Resolution Imaging Techniques for Automotive Radar
- Mon, 7 Oct 2024
- 15:00
- Aula Senaatszaal