How to Create Interactive CLIs in Python?
It was a terrible assumption. I thought that once deployed, it was over. But, deployment is only the beginning for most data science projects.
Frequently, we have to retrain and update the model. Often with new data, sometimes with a different configuration, and occasionally with a unique architecture altogether.
To make things worse, sometimes, you hand it over to another team or a client who doesn’t have the technical skills to do it. If not, whoever is doing these maintenance may have a different understanding of the architecture.
In such cases, we build a web portal to support aftercare. We link the app to the data stores, let the user do configurations through a web form, and run the algorithms.
Building a web app to interact with your machine learning models is not a bad idea. Especially tools such as Streamlit allows data scientists to create web apps without a single line of HTML, CSS, or JavaScript.
Yet, a web app isn’t the right fit for a few. Say you have concerns about hosting the web app. Don’t worry; it’s not a dead end. We have a fallback option that is indeed a robust solution for the problem.
You can create a command-line interface (CLI) to productize and deliver your machine-learning projects.
What can you do with a CLI for your machine-learning model?
CLIs allow you to run programming instructions on a command prompt without interacting with the codebase. CLIs can have many commands, each taking different arguments. For example, the below starts a web server, and you can tell which port to use as an option.
python -m "http.server" 8080
You could create helpful CLIs like this to interact with your ML model as well. If your client wants to retrain the model with different data, they can do so with a single command like the one below.
manage retrain /<path>/new_data.csvYou could also create help pages for your CLI, show progress while training (or any other task,) and meaningfully style logs and errors in the terminal window.
Creating your first CLI in Python
We will be using a Python library called Typer to create CLIs. Typer is a minimal framework to create CLI commands, print stylish output, and show progress.
You can still use Typer for creating CLIs for your non-Python programs. You have to use either a Python subprocess or communicate with your non-Python code through HTTP or a message broker. We won’t discuss them here. But, it may be a topic for a future article.
Installing Typer is straightforward. You can use PyPI:
pip install typerOnce the installation is complete, you can try this Hello World app to get a sense of how Typer works.
Create a file called app.js in your project directory (you can pick a different name) and the below content.
import typer
def say_hello():
typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.BLUE)
if __name__ == "__main__":
typer.run(say_hello)
Running the above command will print a colorful Hello World message on the terminal.
Of course, the above exercise is more than a simple Hello World. You can change the text color and font color. If you are using an editor with IntelliSense, like VSCode, finding them is easy. If not, you can find them on Typer’s documentation, which is worth checking anyway.
Before moving on, let’s also look at how to add multiple commands to your CLI. It needs a slight modification to the codebase.
We create an instance of the Typer and call it in the ‘main’ method. It needs a slight modification to the codebase. We can use the ‘command’ decorator on functions to convert each of them into a CLI command.
import typer
app = typer.Typer()
@app.command()
def say_hello():
typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.BLUE)
@app.command()
def say_hello_in_red():
typer.secho(f"Hello World!", fg=typer.colors.WHITE, bg=typer.colors.RED)
if __name__ == "__main__":
app()
With this new arrangement, you can have more than one command under the same CLI. You can mention the function following the filename to tell the CLI which one to execute.
Isn’t this awesome already? Now that we’ve installed and tested Typer, let’s move on to integrating an ML algorithm.
Creating CLI for K-Means algorithm.
I’ll be using the K-Means algorithm, which I have discussed in a previous post. K-Means is a simple and powerful clustering technique to group data points.
Here is a modified version of our app. We’ve created two commands; one for train and save the K-Means model and one to load and use in predictions. Note that the train function definition takes an argument, file_path. Typer will convert it into a command-line argument.
import typer
import pandas as pd # Pandas for reading data
from sklearn.cluster import KMeans # KMeans Clustering itself
from joblib import dump, load # Required to persist the trained model
app = typer.Typer()
@app.command()
def train(file_path: str):
typer.secho(f"Training K-Means with file {file_path}")
# Code for training K-Means clustering
df = pd.read_csv(file_path)
kmeans = KMeans(n_clusters=2, random_state=0).fit(df[["Age", "Income"]])
typer.secho("CLUSTER CENTERS")
typer.echo(kmeans.cluster_centers_)
dump(
kmeans, "model.joblib"
) # This line is to store the model to use in predictions later.
@app.command()
def predict():
typer.secho("Will be implemented soon")
pass
if __name__ == "__main__":
app()
Running the ‘train’ command of our app CLI with a file path will do the trick. You can practice it for yourself by implementing the predict command.
Showing the progress bar in your CLI
For consuming tasks, you’ll have to show a progress bar so that the user doesn’t have to pull their hair out. The Typer API helps you create progress bars without pulling yours.
Let’s put in place another command to find out the optimal number of clusters using the elbow method. This method will run K-Means many times with different numbers of groups. The ideal number would be the one with low inertia. In large applications, this might be a task running for several hours.
Here is the code snippet that does the job. Note that, this time, we’ve added two arguments and one of them has a default value. Typer will consider it as an option rather than an argument. You may choose to leave it blank.
@app.command()
def elbow(file_path: str, max_clusters:int=10):
errors = [] # Create an empty list to collect inertias
df = pd.read_csv(file_path)
# Loop through some desirable number of clusters
# The KMeans algorithm's fit method returns a property called inertia_ that has the information we need
for k in range(2, max_clusters):
time.sleep(1)
errors.append(
{
"inertia": KMeans(n_clusters=k, random_state=0)
.fit(df[["Age", "Income", "Debt"]])
.inertia_,
"num_clusters": k,
}
)
# for convenience convert the list to a pandas dataframe
df_inertia = pd.DataFrame(errors)
typer.echo(df_inertia)
Now try running python app.py elbow voters_demo_sample.csv.
This code snippet currently doesn’t have a progress bar. Running this will keep the terminal without any output for several seconds and print it all at once.
Let’s go ahead and put a progress bar to help our users. Here is the updated script. Note the slight change in the for-loop.
@app.command()
def elbow(file_path: str, max_clusters: int = 10):
errors = [] # Create an empty list to collect inertias
df = pd.read_csv(file_path)
# Loop through some desirable number of clusters
# The KMeans algorithm's fit method returns a property called inertia_ that has the information we need
with typer.progressbar(range(2, max_clusters)) as progress:
for k in progress:
time.sleep(1)
errors.append(
{
"inertia": KMeans(n_clusters=k, random_state=0)
.fit(df[["Age", "Income", "Debt"]])
.inertia_,
"num_clusters": k,
}
)
# for convenience convert the list to a pandas dataframe
df_inertia = pd.DataFrame(errors)
typer.echo(df_inertia)
Here’s what running this now would look like:
Creating a man page for your CLI.
Man pages are documentation to help the user. Typer is smart to convert your functions and their inputs into detailed man pages. It’ll list out all the available commands, arguments, and options.
You can access the Typer-generated man pages with the suffix – -help.
python app.py --help
You can access the man page of a specific command as well:
python app.py elbow --help
If you need to give the user more information, you can use a multi-line comment at the beginning of the function. Typer will display them on the help page.
@app.command()
def elbow(file_path: str, max_clusters: int = 10):
"""
This command will run K-Means algorithm many times and print the inertia values to the console.
"""
...
Conclusion
In this article, we created a CLI to aid interaction with machine learning models. You can extend these techniques beyond data science projects. A CLI could handle any user interaction.
Although we’ve used Python to generate CLIs, you may use it to run other programs as well. In such cases, you may have to use either subprocess, HTTP, or a message broker.
CLIs are a quick fix for a crucial problem. You can build one if developing a web portal or other solutions aren’t a good fit. Also, you don’t have to worry about unforeseeable events like server downtime. It makes CLIs a robust alternative to consider.
With Typer, a minimalistic Python library, we created
- CLI commands to interact with the K-Means algorithm;
- colorful messages on a terminal window;
- a progress bar to keep the user informed and;
- command-line documentation to educate the user.
Thanks for the read, friend. It seems you and I have lots of common interests. Say Hi to me on LinkedIn, Twitter, and Medium.
Not a Medium member yet? Please use this link to become a member because I earn a commission for referring at no extra cost for you.
