Taxonomy of Real Faults in Deep Learning Systems
Large taxonomy of faults in deep learning (DL) systems.
checklist bugs deep-learning research paper arxiv:1910.11015

We have manually analysed 1059 artefacts gathered from GitHub commits and issues of projects that use the most popular DL frameworks (TensorFlow, Keras and PyTorch) and from related Stack Overflow posts. Structured interviews with 20 researchers and practitioners describing the problems they have encountered in their experience have enriched our taxonomy with a variety of additional faults that did not emerge from the other two sources. Our final taxonomy was validated with a survey involving an additional set of 21 developers, confirming that almost all fault categories (13/15) were experienced by at least 50% of the survey participants.

Don't forget to tag @paolo-tonella in your comment, otherwise they may not be notified.

Authors community post
Share this project
Similar projects
🚧 Simple considerations for simple people building fancy NNs
I will try to highlight a few steps of my mental process when it comes to building and debugging neural networks.
How (And Why) to Create a Good Validation Set
Steps for creating a representative validation set for training.
Checklist for debugging neural networks
Tangible steps you can take to identify and fix issues with training, generalization, and optimization for machine learning models.
Health Checks for Machine Learning
A guide to model retraining and evaluation.
Top collections