Iteration


Accompanying video for this lesson. Subscribe for updates!

Intuition

We don’t want to spend months of time developing a complicated solution only to learn that the entire problem has changed. The main idea here is to close the loop, which involves:

  1. Create a minimum viable product (MVP) that satisfies a baseline performance.
  2. Iterate on your solution by using the feedback.
  3. Constantly reassess to ensure your objective hasn’t changed.
product development cycle
Product development cycle

Creating the MVP for solutions that requires machine learning often involves going manual before ML.

Deploying solutions is actually quite easy (from an engineering POV) but maintaining and iterating upon it is quite the challenge.

Watch from 0:00 for a video walkthrough of this section.

Application

For our solution, we’ll have an initial set of baselines where we’ll start with a rule-based approach and then slowly add complexity (regression → CNN → Transformers).

For the purpose of this course, even our MVP will be an ML model, however we would normally deploy the rule-based approach first as long as it satisfies a performance threshold.

As for monitoring and iterating on our solution, we’ll be looking at things like overall performance, class specific performances, # of relevant tags, etc. We’ll also create workflows to look at new data for anomalies, apply active learning, ease the annotation process, etc.

Watch from 3:08 for a video walkthrough of this section.

Resources


[ product product-management evaluation ]