This is a summary of the models available in the transformers library. It assumes you’re familiar with the original transformer model. For a gentle introduction check the annotated transformer. Here we focus on the high-level differences between the models. You can check them more in detail in their respective documentation. Also checkout the pretrained model page to see the checkpoints available for each type of model.