Search results

Finetune: Scikit-learn Style Model Finetuning for NLP
Finetune is a library that allows users to leverage state-of-the-art pretrained NLP models for a wide variety of downstream tasks.
natural-language-processing finetuning pretraining transformers
Finetuning Transformers with JAX + Haiku
Walking through a port of the RoBERTa pre-trained model to JAX + Haiku, then fine-tuning the model to solve a downstream task.
jax haiku roberta transformers
Cycle Text-To-Image GAN with BERT
Image generation from their respective captions, building on state-of-the-art GAN architectures.
generative-adversarial-networks bert transformers image-to-text
A Survey of Long-Term Context in Transformers
Over the past two years the NLP community has developed a veritable zoo of methods to combat expensive multi-head self-attention.
transformers multi-head-attention attention natural-language-processing
ELECTRA: Pre-training Text Encoders as Discriminators
PyTorch implementation of the electra model from the paper: ELECTRA - Pre-training Text Encoders as Discriminators Rather Than Generators
language-modeling discriminators generators pytorch
Talking-Heads Attention
A variation on multi-head attention which includes linear projections across the attention-heads dimension, immediately before and after the softmax ...
multi-head-attention talking-heads-attention attention transformers
Custom Classifier on Top of Bert-like Language Model
Take pre-trained language model and build custom classifier on top of it.
bert language-modeling pytorch pytorch-lightning
Rethinking Batch Normalization in Transformers
We found that NLP batch statistics exhibit large variance throughout training, which leads to poor BN performance.
power-normalization batch-normalization transformers natural-language-processing
Mention Classifier
Category prediction model This repo contains AllenNLP model for prediction of Named Entity categories by its mentions.
natural-language-processing named-entity-recognition mention-classification tutorial
NLP Research Highlights — Issue #1
First quarterly issue of the natural language processing (NLP) Research Highlights series.
natural-language-processing highlights summaries text-to-text-transformer
projects 1 - 10 of 23
Share your project
Share what you've made with ML. Post Project
Learn practical AI
Learn ML with clean code, simplified math and visuals.
Lessons MWML lessons are among the top 10 ML repositories on GitHub.
Join the community
Get feedback on your projects, interview prep, and more! Chat on Slack