ML in Production - Deployment Series
A multi-part blog series on deploying machine learning models in an automated, reproducible, and auditable manner.
production guide article tutorial

  • Guide 01: What Does it Mean to Deploy a Machine Learning Model? - What does it even mean to "deploy a model?" How does deployment fit into the machine learning process? What factors should you take into consideration when deciding how to deploy?
  • Guide 02: Software Interfaces for Machine Learning Deployment - Deployment is considerably easier when you’re working with the right interfaces. Doubly important when you’re using models across different frameworks and languages. So what’s the right interface to make deployment easier?
  • Guide 03: Batch Inference for Machine Learning Deployment - If you can precompute and cache predictions in batch, DO IT! It’s much easier than deploying and maintaining APIs and other near real time infrastructure. Here’s how to do batch inference.
  • Guide 04: The Challenges of Online Inference - But when you need predictions in real time, you need online inference. There are many gotchas in online inference: you need to query data from multiple sources in real time, you’ll need A/B testing, you need rollout strategies…
  • Guide 05: Online Inference for ML Deployment - If after learning about those challenges you decide you still need online inference, bless your heart. There are a lot of posts on Flask APIs, but that’s the easiest part. You need versioning, autoscaling, and the ability to A/B test models.
  • Guide 06: Model Registries for ML Deployment - Where do you store all these trained models? Where do you track metadata and lineage? How do you retrieve models at inference time? That’s where you’ll need a model registry.
  • Guide 07: Test-Driven Machine Learning Development - It’s not enough to use aggregate metrics to understand model performance. You need to know how the model does on sub-slices of data. You need machine learning unit tests.
  • Guide 08: A/B Testing Machine Learning Models - Just because a model passes its unit tests, doesn’t mean it will move the product metrics. The only way to establish causality is through online validation. Like any other feature, models need to be A/B tested.

Don't forget to tag @lpatruno in your comment, otherwise they may not be notified.

Authors community post
Data Scientist
Share this project
Similar projects
You Trained a Machine Learning Model, Now What?
Three often overlooked parts of this process occur after the model is actually built: model evaluation, deployment, and monitoring.
Machine Learning Deployment: Shadow Mode
“How do I test my new model in production?” One answer, and a method I often employ when initially deploying models, is shadow mode.
Build your first data warehouse with Airflow on GCP
What are the steps in building a data warehouse? What cloud technology should you use? How to use Airflow to orchestrate your pipeline?
Test-Time Data Augmentation
Tutorial on how to properly implement test-time image data augmentation in a production environment with limited computational resources.
Top collections