Skip to Main content Skip to Navigation

Transfer learning of convolutional neural networks for texture synthesis and visual recognition in artistic images

Abstract : In this thesis, we study the transfer of Convolutional Neural Networks (CNN) trained on natural images to related tasks. We follow two axes: texture synthesis and visual recognition in artworks. The first one consists in synthesizing a new image given a reference sample. Most methods are based on enforcing the Gram matrices of ImageNet-trained CNN features. We develop a multi-resolution strategy to take into account large scale structures. This strategy can be coupled with long-range constraints either through a Fourier frequency constraint, or the use of feature maps autocorrelation. This scheme allows excellent high-resolution synthesis especially for regular textures. We compare our methods to alternatives ones with quantitative and perceptual evaluations. In a second axis, we focus on transfer learning of CNN for artistic image classification. CNNs can be used as off-the-shelf feature extractors or fine-tuned. We illustrate the advantage of the last solution. Second, we use feature visualization techniques, CNNs similarity indexes and quantitative metrics to highlight some characteristics of the fine-tuning process. Another possibility is to transfer a CNN trained for object detection. We propose a simple multiple instance method using off-the-shelf deep features and box proposals, for weakly supervised object detection. At training time, only image-level annotations are needed. We experimentally show the interest of our models on six non-photorealistic.
Complete list of metadata
Contributor : ABES STAR :  Contact
Submitted on : Monday, May 17, 2021 - 11:33:09 AM
Last modification on : Tuesday, October 19, 2021 - 11:14:15 AM
Long-term archiving on: : Wednesday, August 18, 2021 - 6:23:52 PM


Version validated by the jury (STAR)


  • HAL Id : tel-03227373, version 1



Nicolas Gonthier. Transfer learning of convolutional neural networks for texture synthesis and visual recognition in artistic images. Computer Vision and Pattern Recognition [cs.CV]. Université Paris-Saclay, 2021. English. ⟨NNT : 2021UPASG024⟩. ⟨tel-03227373⟩



Record views


Files downloads