Self-Supervised Scene De-occlusion
We investigate the problem of scene de-occlusion, which aims to recover the underlying occlusion ordering and complete the invisible parts of occluded ...
self-supervised-learning computer-vision de-occlusion image-generation research tutorial
Resource links
Top collections
Details
Objectives & Highlights

We make the first attempt to address the problem through a novel and unified framework that recovers hidden scene structures without ordering and amodal annotations as supervisions. This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner. Based on PCNet-M and PCNet-C, we devise a novel inference scheme to accomplish scene de-occlusion, via progressive ordering recovery, amodal completion and content completion.

Takeaways & Next Steps

Remarkably, our approach that is trained in a self-supervised manner achieves comparable results to fully-supervised methods. The proposed scene de-occlusion framework benefits many applications, including high-quality and controllable image manipulation and scene recomposition (see Fig. 1), as well as the conversion of existing modal mask annotations to amodal mask annotations.

Don't forget to tag @XiaohangZhan in your comment.

Authors
PhD student in MMLab, CUHK
Share this project
Similar projects
Self-Supervised Learning and Computer Vision
So, what do you do if there are no pre-trained models in your domain?
A Visual Exploration of DeepCluster
DeepCluster is a self-supervised method to combine clustering and representation learning
The Illustrated Self-Supervised Learning
A visual introduction to self-supervised learning methods in Computer Vision
Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset
The largest dataset of in-the-wild stereo image pairs (50K) crowd-sourced from the Holopix lightfield image-sharing social network.