The Sorcerer’s Apprentice Guide to Training LSTMs
Tricks of the trade for training Long Short-Term Memory networks.
recurrent-neural-networks lstm tips article

Last year, I took a course at the Johannes Kepler University in Linz, Austria on the topic of Recurrent Neural Networks and Long Short-Term Memory Networks. There, Sepp Hochreiter shared some of the “magic tricks” he and his team employ for training LSTMs. This blog post is the accumulation of some of my notes.

For this post, I assume you are already familiar with LSTMs. If not, I suggest you begin with Chris Olah’s Understanding LSTM Networks and then go on to read the original LSTM work [1] .

Share this project
Similar projects
Recurrent Neural Networks: building GRU cells VS LSTM cells
What are the advantages of RNN’s over transformers? When to use GRU’s over LSTM? What are the equations of GRU really mean? How to build a GRU cell in ...
Speaker Diarization (CNN, RNN, LSTM)
Speaker Diarization is the problem of separating speakers in audio.
Understanding LSTM Networks
A closer look at the inner workings of LSTM networks.
A Visual Guide to Recurrent Layers in Keras
Understand how to use Recurrent Layers like RNN, GRU and LSTM in Keras with diagrams.
Top collections