Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, EpiSciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Conference papers

Online Continual Learning under Extreme Memory Constraints

Enrico Fini Stéphane Lathuilière 1, 2 Enver Sangineto Moin Nabi Elisa Ricci 
1 MM - Multimédia
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract : Continual Learning (CL) aims to develop agents emulating the human ability to sequentially learn new tasks while being able to retain knowledge obtained from past experiences. In this paper, we introduce the novel problem of Memory-Constrained Online Continual Learning (MC-OCL) which imposes strict constraints on the memory overhead that a possible algorithm can use to avoid catastrophic forgetting. As most, if not all, previous CL methods violate these constraints, we propose an algorithmic solution to MC-OCL: Batch-level Distillation (BLD), a regularization-based CL approach, which effectively balances stability and plasticity in order to learn from data streams, while preserving the ability to solve old tasks through distillation. Our extensive experimental evaluation, conducted on three publicly available benchmarks, empirically demonstrates that our approach successfully addresses the MC-OCL problem and achieves comparable accuracy to prior distillation methods requiring higher memory overhead.
Document type :
Conference papers
Complete list of metadata
Contributor : Stéphane Lathuilière Connect in order to contact the contributor
Submitted on : Thursday, September 17, 2020 - 2:22:46 PM
Last modification on : Wednesday, November 3, 2021 - 6:18:14 AM

Links full text


  • HAL Id : hal-02941923, version 1
  • ARXIV : 2008.01510



Enrico Fini, Stéphane Lathuilière, Enver Sangineto, Moin Nabi, Elisa Ricci. Online Continual Learning under Extreme Memory Constraints. European Conference on Computer Vision, Aug 2020, edinburgh, United Kingdom. ⟨hal-02941923⟩



Record views