• Taking existing pre-trained language model and understanding it’s output - here I use PolBERTa trained for Polish language. • Building custom classification head on top of the LM. • Using fast tokenizers to efficiently tokenize and pad input text as well as prepare attention masks. • Preparing reproducible training code with PyTorch Lightning. • Finding good starting learning rate for the model. • Validating the trained model on PolEmo 2.0 dataset (benchmark for Polish language sentiment analysis with 4 classes).
Don't forget to add the tag @marrrcin in your comments.