HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels

Abstract : Contrastive Learning has shown impressive results on natural and medical images, without requiring annotated data. However, a particularity of medical images is the availability of meta-data (such as age or sex) that can be exploited for learning representations. Here, we show that the recently proposed contrastive y-Aware InfoNCE loss, that integrates multi-dimensional meta-data, asymptotically optimizes two properties: conditional alignment and global uniformity. Similarly to [33], conditional alignment means that similar samples should have similar features, but conditionally on the meta-data. Instead, global uniformity means that the (normalized) features should be uniformly distributed on the unit hypersphere, independently of the meta-data. Here, we propose to define conditional uniformity, relying on the meta-data, that repel only samples with dissimilar metadata. We show that direct optimization of both conditional alignment and uniformity improves the representations, in terms of linear evaluation, on both CIFAR-100 and a brain MRI dataset.
Complete list of metadata

https://hal.telecom-paris.fr/hal-03523114
Contributor : Pietro Gori Connect in order to contact the contributor
Submitted on : Wednesday, January 12, 2022 - 2:46:15 PM
Last modification on : Thursday, January 27, 2022 - 3:45:13 AM
Long-term archiving on: : Wednesday, April 13, 2022 - 11:05:25 PM

File

2111.05643.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03523114, version 1

Citation

Benoit Dufumier, Pietro Gori, Julie Victor, Antoine Grigis, Edouard Duchesnay. Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels. Med-NeurIPS - Workshop NeurIPS, Dec 2021, Vancouver, Canada. ⟨hal-03523114⟩

Share

Metrics

Record views

43

Files downloads

14