Command Line Interface (CLI) Applications

Using a command line interface (CLI) application to organize our application's processes.

Goku Mohandas

📬 Receive new lessons straight to your inbox (once a month) and join 35K+ developers in learning how to responsibly deliver value with ML.

Intuition

When it comes to serving our models, we need to think about exposing the application's functionality to ourselves, team members and ultimately our end users. And the interfaces to achieve this will be different. Recall from our Organization lesson where we executed our main operations via the terminal and Python interpreter.

from tagifai import main
main.etl_data()

or the alternative was to call the operation explicitly inside our main.py file:

# tagifai/main.py
if __name__ == "__main__":
    etl_data()

This becomes extremely tedious having to dig into the code and execute functions one at a time. A solution is to build a command line interface (CLI) application that allows for interaction at the operational level. It should designed such that we can view all possible operations (and their required arguments) and execute them from the shell.

Application

We're going to create our CLI using Typer:

pip install typer==0.4.1

# Add to requirements.txt
typer==0.4.1

It's as simple as initializing the app and then adding the appropriate decorator to each function operation we wish to use as a CLI command in our main.py:

1
2
3

# tagifai/main.py
import typer
app = typer.Typer()

1
2
3

@app.command()
def etl_data():
    ...

We'll repeat the same for all the other functions we want to access via the CLI: etl_data(), label_data(), train_model(), optimize(), predict_tag(). We'll make it such that all arguments are optional so that we can explicitly define them in our bash commands. For example, def label_data(args_fp: str): will become def label_data(args_fp: str = "config/args.json"):.

View tagifai/main.py function headers

@app.command()
def etl_data():
    ...

@app.command()
def label_data(args_fp: str = "config/args.json") -> None:
    ...

@app.command()
def train_model(
    args_fp: str = "config/args.json",
    experiment_name: str = "baselines",
    run_name: str = "sgd",
    test_run: bool = False,
) -> None:
    ...


@app.command()
def optimize(
    args_fp: str = "config/args.json",
    study_name: str = "optimization",
    num_trials: int = 20
) -> None:
    ...

@app.command()
def predict_tag(text: str = "", run_id: str = None) -> None:
    ...

Commands

To use our CLI app, we can first view the available commands thanks to the decorators we added to certain functions we wanted to expose to the CLI:

python tagifai/main.py --help

Typer also comes with a utility tool called typer-cli but there are some dependency conflicts with our other libraries so we won't be using it.

Usage: main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help       Show this message and exit.

Commands:
  etl-data     Extract, transform and load data.
  label-data   Label data using constraints.
  optimize     Optimize hyperparameters.
  predict-tag  Predict tag for text.
  train-model  Train a model given arguments.

Arguments

With Typer, a function's input arguments automatically get rendered as command line options. For example, our predict_tags function consumes text and an optional run_id as inputs which automatically become arguments for the predict-tags CLI command.

@app.command()
def predict_tag(text: str = "", run_id: str = None) -> None:
    """Predict tag for text.

    Args:
        text (str): input text to predict label for.
        run_id (str, optional): run id to load artifacts for prediction. Defaults to None.
    """
    if not run_id:
        run_id = open(Path(config.CONFIG_DIR, "run_id.txt")).read()
    artifacts = load_artifacts(run_id=run_id)
    prediction = predict.predict(texts=[text], artifacts=artifacts)
    logger.info(json.dumps(prediction, indent=2))
    return prediction

But we can also ask for help with this specific command without having to go into the code:

python tagifai/main.py predict-tag --help

Usage: main.py predict-tag [OPTIONS]

  Predict tag for text.

  Args:
    text (str): input text to predict label for.
    run_id (str, optional): run id to load artifacts for prediction. Defaults to None.

Options:
  --text TEXT
  --run-id TEXT
  --help         Show this message and exit.

Usage

Finally, we can execute the specific command with all the arguments:

python tagifai/main.py predict-tag --text="Transfer learning with transformers for text classification."

[
    {
        "input_text": "Transfer learning with transformers for text classification.",
        "predicted_tag": "natural-language-processing"
    }
]

To cite this lesson, please use:

@article{madewithml,
    author       = {Goku Mohandas},
    title        = { Command line - Made With ML },
    howpublished = {\url{https://madewithml.com/}},
    year         = {2021}
}