Epipolar Transformers
Differentiable "epipolar transformer", which enables the 2D detector to leverage 3D-aware features to improve 2D pose estimation.
pose-estimation 3d epipoplar transformers computer-vision natural-language-processing tutorial research cvpr-2020 code paper video arxiv:2005.04551

The intuition is: given a 2D location p in the current view, we would like to first find its corresponding point p' in a neighboring view, and then combine the features at p' with the features at p, thus leading to a 3D-aware feature at p. Inspired by stereo matching, the epipolar transformer leverages epipolar constraints and feature matching to approximate the features at p'.

Don't forget to tag @yihui-he in your comment, otherwise they may not be notified.

Authors community post
master@CMU focused on CV & DL. I open-source as much as possible.
Share this project
Similar projects
OpenMMLab Computer Vision
MMCV is a python library for CV research and supports many research projects such as object detection, segmentation, pose estimation, action ...
Face Alignment in Full Pose Range: A 3D Total Solution
Face Alignment in Full Pose Range: A 3D Total Solution.
V2V-PoseNet Pytorch
PyTorch implementation of V2V-PoseNet with IntegralPose/PoseFix loss.
Pose Animator
Takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh.