Last year, I took a course at the Johannes Kepler University in Linz, Austria on the topic of Recurrent Neural Networks and Long Short-Term Memory Networks. There, Sepp Hochreiter shared some of the “magic tricks” he and his team employ for training LSTMs. This blog post is the accumulation of some of my notes.

For this post, I assume you are already familiar with LSTMs. If not, I suggest you begin with Chris Olah’s Understanding LSTM Networks and then go on to read the original LSTM work [1] .

Recurrent Neural Networks: building GRU cells VS LSTM cells

What are the advantages of RNN’s over transformers? When to use GRU’s over LSTM? What are the equations of GRU really mean? How to build a GRU cell in ...

Speaker Diarization (CNN, RNN, LSTM)

Speaker Diarization is the problem of separating speakers in audio.

Understanding LSTM Networks

2015-08-27 ·
A closer look at the inner workings of LSTM networks.

A Visual Guide to Recurrent Layers in Keras

2020-04-23 ·
Understand how to use Recurrent Layers like RNN, GRU and LSTM in Keras with diagrams.