Summary of 🤗 Transformers Models
A high-level summary of the differences between each model in HuggingFace's Transformer library.
transformers huggingface summary natural-language-processing tutorial article

This is a summary of the models available in the transformers library. It assumes you’re familiar with the original transformer model. For a gentle introduction check the annotated transformer. Here we focus on the high-level differences between the models. You can check them more in detail in their respective documentation. Also checkout the pretrained model page to see the checkpoints available for each type of model.

Don't forget to tag @huggingface , @sgugger in your comment, otherwise they may not be notified.

Authors community post
Solving NLP, one commit at a time!
Research Engineer at HuggingFace
Share this project
Similar projects
Project Insight is designed to create NLP as a service with code base for both front end GUI (streamlit) and backend server (FastAPI) the usage of ...
Transformers - Hugging Face
🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.
Top Down Introduction to BERT with HuggingFace and PyTorch
I will also provide some intuition into how BERT works with a top down approach (applications to algorithm).
CodeBERT - Masked Language Model for source code
Tutorial to use codeBERT a MLM for Python code. Model trained from scratch using roBERTa
Top collections