Self-Supervision with FastAI
A tutorial of rotation-based self-supervision using FastAI2 & PyTorch!
self-supervised-learning fastai pytorch tutorial article code notebook

Using self-supervision can help learn features that can transfer to a down-stream task, such as classification! In this example, we used rotation predication as our pretext task for feature representation learning. Pretraining our model on rotation prediction prior to training for classification allowed us to achieve 61.7% accuracy, on just 0.3% of the labeled data (180 samples)! Training from scratch with the same amount of data yields an accuracy of 13%. The motivation for using self-supervised learning is the ability to train models with decent accuracy without the need of much labeled data!

Don't forget to tag @AmarSaini in your comment.

Deep Learning @ LLNL || M.S. - Electrical Engineering and Computer Science || B.S. - Computer Science & Engineering || University of California, Merced
Share this project
Similar projects
Self-Supervised Scene De-occlusion
We investigate the problem of scene de-occlusion, which aims to recover the underlying occlusion ordering and complete the invisible parts of occluded ...
Self-Supervised Learning -- UC Berkeley Spring 2020
Lecture on self-supervised learning from CS294-158-SP20: Deep Unsupervised Learning.
A Visual Exploration of DeepCluster
DeepCluster is a self-supervised method to combine clustering and representation learning
Plan2Explore: Plan to Explore via Self-Supervised World Models
A self-supervised reinforcement learning agent that tackles task-specific and the sample efficiency challenges.