Skip to content

CI/CD Workflows

Using workflows to establish continuous integration and delivery pipelines to reliably iterate on our application.
Goku Mohandas
· ·

📬  Receive new lessons straight to your inbox (once a month) and join 30K+ developers in learning how to responsibly deliver value with ML.


Continuous integration (CI) allows our team to develop, test and integrate code in a structured fashion. This allows the team to more confidently and frequently develop since their work will be properly integrated. Continuous delivery (CD) is responsible for delivering our integrated code to a variety of applications that are dependent on it. With CI/CD pipelines, we can develop and deploy knowing that our systems can quickly adapt and work as intended.

GitHub Actions

There are many tooling options for when it comes to creating our CI/CD pipelines, such as Jenkins, TeamCity, CircleCI and many others. However, we're going to use GitHub Actions to create automatic workflows to setup our CI/CD pipelines.

GitHub Actions has the added advantage of integrating really well with GitHub and since all of our work is versioned there, we can easily create workflows based on GitHub events (push, PR, release, etc.). GitHub Actions also has a rich marketplace full of workflows that we can use for our own project. And, best of all, GitHub Actions is free for public repositories and actions using self-hosted runners (running workflows on your own hardware or on the cloud).


We'll learn about GitHub Actions by understanding the components that compose an Action. These components abide by a specific workflow syntax which can be extended with the appropriate context and expression syntax.


With GitHub Actions, we are creating automatic workflows to do something for us. For example, this testing workflow is responsible for conducting tests on our code base. We can specify the name of our workflow at the top of our YAML file.

# .github/workflows/testing.yml
name: testing


Workflows are triggered by an event, which can be something that occurs on a schedule (cron), webhook or manually. In our application, we'll be using the push and pull request webhook events to run the testing workflow when someone directly pushes or submits a PR to the main branch.

# .github/workflows/testing.yml
    - main
    - main


Be sure to check out the complete list of the different events that can trigger a workflow.


Once the event is triggered, a set of jobs run on a runner, which is the application that runs the job using a specific operating system. Our first (and only) job is test-code which runs on the latest version of ubuntu.

# .github/workflows/testing.yml
    runs-on: ubuntu-latest


Jobs run in parallel but if you need to create dependent jobs, where if a particular job fails all it's dependent jobs will be skipped, then be sure to use the needs key. One a similar note, we can also share data between jobs.


Each job contains a series of steps which are executed in order. Each step has a name, as well as actions to use from the GitHub Action marketplace or commands we want to run. For the test-code job, the steps are to checkout the repo, install the necessary dependencies and run tests.

# .github/workflows/testing.yml
    runs-on: ubuntu-latest
      - name: Checkout repo
        uses: actions/[email protected]
      - name: Set up Python
        uses: actions/[email protected]
          python-version: 3.7.10
      - name: Caching
        uses: actions/[email protected]
          path: $/{/{ env.pythonLocation /}/}
          key: $/{/{ env.pythonLocation /}/}-$/{/{ hashFiles('') /}/}-$/{/{ hashFiles('requirements.txt') /}/}
      - name: Install dependencies
        run: python -m pip install -e ".[test]" --no-cache-dir
      - name: Run tests
        run: pytest tests/tagifai --cov tagifai --cov-report html


Notice that one of our steps is to cache the entire Python environment with a specific key. This will significantly speed up the time required to run our Action the next time as long as the key remains unchanged (same python location, and requirements.txt).


Recall that workflows will be triggered when certain events occur. For example, our testing workflow will initiate on a push or PR to the main branch. We can see the workflow's runs (current and previous) on the Actions tab on our repository page. And if we click on a specific run, we can view the all the steps and their outputs as well.

However, when we're first creating a GitHub Action, we may not get our workflow to run so smoothly. Instead of creating multiple commits (if you do, at least squash them together), we can use act to run and test workflows locally. Act executes the jobs defined in our workflows by spinning up a container with an image that is very similar to the image that GitHub's runners use. It has the same environment specifications, variables, etc. and you can inspect both the images and running containers with the respective docker commands (docker images, docker ps).

# List of jobs
$ act -l
ID          Stage  Name
build-docs  0      build-docs
test-code   0      test-code

With act, we can run all the jobs, a specific job, jobs for a specific event (ex. PR), etc.

# Running GitHub Actions locally
act  # Run default PUSH event
act -l  # list all jobs
act pull_request  # PR event
act -j test-code  # specific job
[testing/test-code] 🚀  Start image=catthehacker/ubuntu:act-latest
[testing/test-code]   🐳  docker run image=catthehacker/ubuntu:act-latest platform=linux/amd64 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[testing/test-code]   🐳  docker cp src=/Users/goku/Documents/madewithml/mlops/. dst=/Users/goku/Documents/madewithml/mlops
[testing/test-code] ⭐  Run Checkout repo
[testing/test-code]   ✅  Success - Checkout repo
[testing/test-code] ⭐  Run Execute tests
[testing/test-code]   ✅  Success - Execute tests


While act is able to very closely replicate GitHub's runners there are still a few inconsistencies. For example caching still needs to be figured out with a small HTTP server when the local container is spun up. So if we have a lot of requirements, it might be faster just to experience using GitHub's runners and squashing the commits once we get the workflow to run.


There are a wide variety of GitHub actions available for deploying and serving our ML applications after all the integration tests have passed. Most of them will require that we have a Dockerfile defined that will load and launch our service with the appropriate artifacts. Read more about ML deployment infrastructure in our lesson.


If we want to deploy and serve multiple models at a time, it's highly recommended to use a purpose-built model server such as MLFlow, TorchServe, RedisAI or Nvidia's Triton inference server. These servers have a registry with an API layer to seamlessly inspect, update, serve, rollback, etc. multiple versions of models.

The specific deployment method we use it entirely up dependent on the application, team, existing infrastructure, etc. The key component is that we are able to update our application when all the integration tests pass without having to manually intervene for deployment.


So what exactly are these actions that we're using from the marketplace? For example, our first step in the test-code job above is to checkout the repo using the actions/[email protected] GitHub Action. The Action's link contains information about how to use it, scenarios, etc.

The Marketplace has actions for a variety of needs, ranging from continuous deployment for various cloud providers, code quality checks, etc. Below are a few GitHub Actions that we highly recommend.

  • Great Expectations: ensure that our GE checkpoints pass when any changes are made that could affect the data engineering pipelines. This action also creates a free GE dashboard with Netlify that has the updated data docs.
  • Continuous ML: train, evaluate and monitor your ML models and generate a report summarizing the findings. I personally use this GitHub Action for automatic training jobs on cloud infrastructure (AWS/GCP) or self hosted runners when a change triggers the training pipeline, as opposed to working with Terraform.


Don't restrict your workflows to only what's available on the Marketplace or single command operations. We can do things like include code coverage reports, deploy an updated Streamlit dashboard and attach it's URL to the PR, deliver (CD) our application to an AWS Lambda / EC2, etc.


To cite this lesson, please use:

    author       = {Goku Mohandas},
    title        = { CI/CD workflows - Made With ML },
    howpublished = {\url{}},
    year         = {2021}