Self-Supervision with FastAI
A tutorial of rotation-based self-supervision using FastAI2 & PyTorch!
self-supervised-learning fastai pytorch tutorial article code notebook

Using self-supervision can help learn features that can transfer to a down-stream task, such as classification! In this example, we used rotation predication as our pretext task for feature representation learning. Pretraining our model on rotation prediction prior to training for classification allowed us to achieve 61.7% accuracy, on just 0.3% of the labeled data (180 samples)! Training from scratch with the same amount of data yields an accuracy of 13%. The motivation for using self-supervised learning is the ability to train models with decent accuracy without the need of much labeled data!

Don't forget to tag @AmarSaini in your comment, otherwise they may not be notified.

Deep Learning @ LLNL || M.S. - Electrical Engineering and Computer Science || B.S. - Computer Science & Engineering || University of California, Merced
Share this project
Similar projects
Self Supervised Learning Fastai Extension
Implementation of popular SOTA self-supervised learning algorithms as Fastai Callbacks.
Single-Stage Semantic Segmentation from Image Labels
We attain competitive results by training a single network model for segmentation in a self-supervised fashion using only image-level annotations
Prototypical Contrastive Learning (PCL) for Unsupervised Learning
Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that addresses the fundamental limitations of the popular ...
Contrastic Learner
A simple to use pytorch wrapper for contrastive self-supervised learning on any neural network.