How to Create Interactive CLIs in Python?

July 26, 2021

How to Create Interactive CLIs in Python?

It was a terrible assumption. I thought once deployed, it’s over. But, deployment is only the beginning for most data science projects.

Frequently, we have to retrain and update the model. Often with new data, sometimes with a different configuration, occasionally with a unique architecture altogether.

To make things worse, sometimes, you hand it over to another team or the client who doesn’t have the technical skills to do it. If not, whoever is doing these maintenances may have a different understanding of the architecture.

In such cases, we build a web portal to support aftercare. We link the app to the datastores, let the user do configurations through a web form, and run the algorithms.

Building a web app to interact with your machine learning models is not a bad idea. Especially, tools such as Streamlit allows data scientists to create web apps without a single line of HTML, CSS, or JavaScript.

Related:How to Create Stunning Websites for Your Data Science Projects

Yet, a web app isn’t the right fit for a few. Say you have concerns about hosting the web app. Don’t worry; it’s not a dead end. We have a fallback option that is indeed a robust solution for the problem.

You can create a command-line interface (CLI) to productize and deliver your machine learning projects.

What can you do with a CLI for your machine learning model?

CLIs allow you to run programming instructions on a command prompt without interacting with the codebase. CLIs can have many commands, each taking different arguments. For example, the below starts a web server, and you can tell which port to use as an option.

Python CLI example

python -m "http.server" 8080

You could create helpful CLIs like this to interact with your ML model as well. If your client wants to retrain the model with different data, they can do so with a single command like the one below.

manage retrain /<path>/new_data.csv

You could also create help pages for your CLI, show progress while training (or any other task,) and meaningfully styled logs and errors in the terminal window.

Related: Use Pipe Operations in Python for More Readable and Faster Coding

Creating your first CLI in Python

We will be using a python library called Typer to create CLI’s. Typer is a minimal framework to create CLI commands, print stylish output, and show progress.

You can still use Typer for creating CLI’s for your non Python programs. You have to use either a Python subprocess or communicate with your non-Python code through HTTP or a message broker. We won’t discuss them here. But, it may be a topic for a future article.

Installing Typer is straightforward. You can use PyPI:

pip install typer

Once the installation is complete, you can try this Hello World app to get a sense of how Typer works.

Create a file called app.js in your project directory (you can pick a different name) and the below content.

import typer

def say_hello():
    typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.BLUE)

if __name__ == "__main__":
    typer.run(say_hello)

Running the above command will print a colorful Hello World message on the terminal.

Running CLI created with Python on a command prompt

Of course, the above exercise is more than a simple Hello World. You can change the text color and font color. If you are using an editor with IntelliSense, like VSCode, finding them is easy. If not, you can find them on Typer’s documentation, which is worth checking anyway.

Typer has several options to format CLI ouputs. They are accessible via autosuggetion by any IDE that supports Python.

Before moving on, let’s also look at how to add multiple commands to your CLI. It needs a slight modification to the codebase.

We create an instance of the Typer and call it in the ‘main’ method. It needs a slight modification to the codebase. We can use the ‘command’ decorator on functions to convert each of them into a CLI command.

import typer

app = typer.Typer()


@app.command()
def say_hello():
    typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.BLUE)


@app.command()
def say_hello_in_red():
    typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.RED)


if __name__ == "__main__":
    app()

With this new arrangement, you can have more than one command under the same CLI. You can mention the function following the filename to tell the CLI which one to execute.

The Python library for CLI creation, Typer, supports colorful outputs in the command prompt

Isn’t this awesome already? Now that we’ve installed and tested Typer let’s move on to integrating an ML algorithm.

Creating CLI for K-Means algorithm.

I’ll be using the K-Means algorithm, which I have discussed in a previous post. K-Means is a simple and powerful clustering technique to group data points.

Related: Running Election Campaigns With K-Means Clustering.

Here is a modified version of our app. We’ve created two commands; one for train and save the K-Means model and one to load and use in predictions. Note that the train function definition takes an argument, file_path. Typer will convert it into a command-line argument.

import typer

import pandas as pd  # Pandas for reading data
from sklearn.cluster import KMeans  # KMeans Clustering itself
from joblib import dump, load  # Required to persist the trained model

app = typer.Typer()


@app.command()
def train(file_path: str):
    typer.secho(f"Training K-Means with file {file_path}")

    # Code for training K-Means clustering
    df = pd.read_csv(file_path)
    kmeans = KMeans(n_clusters=2, random_state=0).fit(df[["Age", "Income"]])

    typer.secho("CLUSTER CENTERS")
    typer.echo(kmeans.cluster_centers_)

    dump(
        kmeans, "model.joblib"
    )  # This line is to store the model to use in predictions later.


@app.command()
def predict():
    typer.secho("Will be implemented soon")
    pass


if __name__ == "__main__":
    app()

Running the ‘train’ command of our app CLI with a file path will do the trick. You can practice it for yourself by implementing the predict command.

This is an example CLI created in Python to train K-Means clustering

Showing progress bar in your CLI

For consuming tasks, you’ll have to show a progress bar so that the user doesn’t have to pull their hair out. The Typer API helps you crate progress bars without pulling yours.

Let’s put in place another command to find out the optimal number of clusters using the elbow method. This method will run K-Means many times with different numbers of groups. The ideal number would be the one with low inertia. In large applications, this might be a task running for several hours.

Here is the code snippet that does the job. Note that, this time, we’ve added two arguments, and one of them has a default value. Typer will consider it as an option rather than an argument. You may choose to leave it blank.

@app.command()
def elbow(file_path: str, max_clusters:int=10):
    errors = []  # Create an empty list to collect inertias

    df = pd.read_csv(file_path)

    # Loop through some desirable number of clusters
    # The KMeans algorithm's fit method returns a property called inertia_ that has the information we need
    for k in range(2, max_clusters):
        time.sleep(1)
        errors.append(
            {
                "inertia": KMeans(n_clusters=k, random_state=0)
                .fit(df[["Age", "Income", "Debt"]])
                .inertia_,
                "num_clusters": k,
            }
        )

    # for convenience convert the list to a pandas dataframe
    df_inertia = pd.DataFrame(errors)

    typer.echo(df_inertia)

Now try running python app.py elbow voters_demo_sample.csv .

Running K-Means elbow method in a CLI

This code snippet currently doesn’t have a progress bar. Running this will keep the terminal without any output for several seconds and print it all at once.

Let’s go ahead and put a progress bar to help our users. Here is the updated script. Note the slight change in the for-loop.

@app.command()
def elbow(file_path: str, max_clusters: int = 10):
    errors = []  # Create an empty list to collect inertias

    df = pd.read_csv(file_path)

    # Loop through some desirable number of clusters
    # The KMeans algorithm's fit method returns a property called inertia_ that has the information we need
    with typer.progressbar(range(2, max_clusters)) as progress:
        for k in progress:
            time.sleep(1)
            errors.append(
                {
                    "inertia": KMeans(n_clusters=k, random_state=0)
                    .fit(df[["Age", "Income", "Debt"]])
                    .inertia_,
                    "num_clusters": k,
                }
            )

    # for convenience convert the list to a pandas dataframe
    df_inertia = pd.DataFrame(errors)

    typer.echo(df_inertia)

Here’s what running this now would look like:

Progress bar in CLI created using Python

Creating a man page for your CLI.

Man pages are documentation to help the user. Typer is smart to convert your functions and their inputs into detailed man pages. It’ll list out all the available commands, arguments, and options.

You can access the Typer-generated man pages with the suffix - -help.

python app.py --help

Documentation generated by Typer CLI library

You can access the man page of a specific command as well:

python app.py elbow --help

CLI documentation for specific command generated by Typer

If you need to give the user more information, you can use a multi-line comment at the beginning of the function. Typer will display them on the help page.

@app.command()
def elbow(file_path: str, max_clusters: int = 10):
    """
    This command will run K-Means algorithm many times and print the inertia values to the console.
    """
    ...

Providing additional CLI documentation using Python docstrings

Conclusion

In this article, we created a CLI to aid interaction with machine learning models. You can extend these techniques beyond data science projects. A CLI could handle any user interaction.

Although we’ve used Python to generate CLI’s, you may use it to run other programs as well. In such cases, you may have to use either subprocess, HTTP, or a message broker.

CLI’s are a quick fix for a crucial problem. You can build one if developing a web portal or other solutions aren’t a good fit. Also, you don’t have to worry about unforeseeable events like server downtime. It makes CLIs a robust alternative to consider.

With Typer, a minimalistic Python library, we created

  • CLI commands to interact with K-Means algorithm;
  • colorful messages on a terminal window;
  • a progress bar to keep the user informed, and;
  • command-line documentation to educate the user.

Related:7 Ways to Make Your Python Project Structure More Elegant


Thanks for reading, friend! Say Hi to me on LinkedIn, Twitter, and Medium.

Not a Medium member yet? Please use this link to become a member because, at no extra cost for you, I earn a small commission for referring you.

How we work

Readers support The Analytics Club. We earn through display ads. Also, when you buy something we recommend, we may get an affiliate commission. But it never affects your price or what we pick.

Connect with us