A Recipe for Training Neural Networks
The most common neural net mistakes and listing a few common gotchas related to training neural nets.
checklist training debugging article recipe tutorial

However, instead of going into an enumeration of more common errors or fleshing them out, I wanted to dig a bit deeper and talk about how one can avoid making these errors altogether (or fix them very fast). The trick to doing so is to follow a certain process, which as far as I can tell is not very often documented. Let’s start with two important observations that motivate it.

Don't forget to tag @karpathy in your comment, otherwise they may not be notified.

Authors
I like to train Deep Neural Nets on large datasets.
Share this project
Similar projects
🚧 Simple considerations for simple people building fancy NNs
I will try to highlight a few steps of my mental process when it comes to building and debugging neural networks.
Data Project Checklist
There’s a lot more to creating useful data projects than just training an accurate model!
The Machine Learning Reproducibility Checklist
How the AI Community Can Get Serious About Reproducibility
Health Checks for Machine Learning
A guide to model retraining and evaluation.
Top collections