12 Factors of Reproducible Machine Learning in Production
We took our experience to deduce 12 factors (as a nod to the 12 factor app) that build the backbone of successful ML in production.
production reproducability machine-learning article code

We’ve faced all of these issues, and more, and now took our experience to deduce 12 factors (as a nod to the 12 factor app) that build the backbone of successful ML in production.

  • Versioning
    • You need to version your code, and you need to version your data.
  • Explicit feature dependencies
    • Make your feature dependencies explicit in your code.
  • Descriptive training and preprocessing
    • Write readable code and separate code from configuration.
  • Reproducibility of trainings
    • Use pipelines and automation.
  • Testing
    • Test your code, test your models.
  • Drift / Continuous training
    • If you data can change run a continuous training pipeline.
  • Tracking of results
    • Track results via automation.
  • Experimentation vs Production models
    • Notebooks are not production-ready, so experiment in pipelines early on.
  • Training-Serving-Skew
    • Correctly embed preprocessing to serving, and make sure you understand up- and downstream of your data.
  • Comparability
    • Build your pipelines so you can easily compare training results across pipelines.
  • Monitoring
    • Again: you build it, you run it. Monitoring models in production is a part of data science in production.
  • Deployability of Models
    • Every training pipeline needs to produce a deployable artefact, not “just” a model.

Don't forget to tag @benkoller in your comment, otherwise they may not be notified.

Authors community post
Share this project
Similar projects
Build your first data warehouse with Airflow on GCP
What are the steps in building a data warehouse? What cloud technology should you use? How to use Airflow to orchestrate your pipeline?
How to Serve ML Models with TensorFlow Serving and Docker
In this tutorial, I’m going to show you how to serve ML models using Tensorflow Serving, an efficient, flexible, high-performance serving system for ML ...
Why Data Quality is Key to Successful ML Ops
A look at ML Ops and highlight how and why data quality is key to ML Ops workflows.
Feature Stores for ML
List of production ML groups and their open-source feature store architectures.