Skip to Main content Skip to Navigation
Journal articles

Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content

Iulia Mitrica 1, 2 Eric Mercier Christophe Ruellan Attilio Fiandrotti 1, 2 Marco Cagnazzo 1, 2 Béatrice Pesquet-Popescu 1, 2 
2 MM - Multimédia
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract : This work addresses the problem of encoding the video generated by the screen of an airplane cockpit. As other computer screens, cockpit screens consists in computer-generated graphics often atop natural background. Existing screen content coding schemes fail notably in preserving the readability of textual information at the low bitrates required in avionic applications. We propose a screen coding scheme where textual information is encoded according to the relative semantics rather than in the pixel domain. The encoder localizes textual information, the semantics of each character are extracted with a convolutional neural network and are predictively encoded. Text is then removed via inpainting, the residual background video is compressed with a standard codec and transmitted to the receiver together with the text semantics. At the decoder side, text is synthesized using the decoded semantics and superimposed over the decoded residual video recovering the original frame. Our proposed scheme offers two key advantages over a semantics-unaware scheme that encodes text in the pixel domain. First, the text readability at the decoder is not compromised by compression artifacts, whereas the relative bitrate is negligible. Second, removal of high-frequency transform coefficients associated to the inpainted text drastically reduces the bitrate of the residual video. Experiments with real cockpit video sequences show BD-rate gains up to 82% and 69 % over a reference H.265/HEVC encoder and its SCC extension. Moreover, our scheme achieves quasi-errorless character recognition already at very low bitrates, whereas even HEVC-SCC needs at least 3 or 4 times more bit-rate to achieve a comparable error rate.
Document type :
Journal articles
Complete list of metadata

Cited literature [44 references]  Display  Hide  Download
Contributor : Cagnazzo Marco Connect in order to contact the contributor
Submitted on : Wednesday, April 24, 2019 - 4:35:28 PM
Last modification on : Wednesday, November 3, 2021 - 6:23:17 AM


Explicit agreement for this submission




Iulia Mitrica, Eric Mercier, Christophe Ruellan, Attilio Fiandrotti, Marco Cagnazzo, et al.. Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content. IEEE Transactions on Multimedia, Institute of Electrical and Electronics Engineers, 2019, 21 (9), pp.2157 - 2170. ⟨10.1109/TMM.2019.2900168⟩. ⟨hal-02109209⟩



Record views


Files downloads