{"id":301,"date":"2021-07-26T00:00:00","date_gmt":"2021-07-26T00:00:00","guid":{"rendered":"https:\/\/tac.debuzzify.com\/?p=301"},"modified":"2023-06-27T23:57:15","modified_gmt":"2023-06-27T23:57:15","slug":"python-cli-tutorial","status":"publish","type":"post","link":"https:\/\/www.the-analytics.club\/python-cli-tutorial\/","title":{"rendered":"How to Create Interactive CLIs in Python?"},"content":{"rendered":"\n\n\n

It was a terrible assumption. I thought that once deployed, it was over. But, deployment is only the beginning for most data science projects.<\/p>\n\n\n\n

Frequently, we have to retrain and update the model. Often with new data, sometimes with a different configuration, and occasionally with a unique architecture altogether.<\/p>\n\n\n\n

To make things worse, sometimes, you hand it over to another team or a client who doesn\u2019t have the technical skills to do it. If not, whoever is doing these maintenance may have a different understanding of the architecture.<\/p>\n\n\n\n

In such cases, we build a web portal to support aftercare. We link the app to the data stores, let the user do configurations through a web form, and run the algorithms.<\/p>\n\n\n\n

Building a web app to interact with your machine learning models<\/a> is not a bad idea. Especially tools such as Streamlit allows data scientists to create web apps<\/a> without a single line of HTML, CSS, or JavaScript.<\/p>\n\n\n\n

Yet, a web app isn\u2019t the right<\/a> fit for a few. Say you have concerns about hosting the web app<\/a>. Don\u2019t worry; it\u2019s not a dead end. We have a fallback option that is indeed a robust solution for the problem.<\/p>\n\n\n\n

You can create a command-line interface (CLI) to productize and deliver your machine-learning projects.<\/p>\n\n\n\n

\n
\n
\n

Grab your aromatic coffee <\/a>(or tea<\/a>) and get ready…!<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

What can you do with a CLI for your machine-learning model?<\/h2>\n\n\n\n

CLIs allow you to run programming instructions on a command<\/a> prompt without interacting with the codebase. CLIs can have many commands, each taking different arguments. For example, the below starts a web server, and you can tell which port to use as an option.<\/p>\n\n\n\n

<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
python<\/span> <\/span>-m<\/span> <\/span>"<\/span>http.server<\/span>"<\/span> <\/span>8080<\/span><\/span>\n<\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n
\n
\"Starting<\/figure><\/div>\n\n\n
You could create helpful CLIs like this to interact with your ML model as well. If your client wants to retrain the model with different data, they can do so with a single command like the one below.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
manage<\/span> <\/span>retrain<\/span> <\/span>\/<\/span><<\/span>pat<\/span>h<\/span>><\/span>\/new_data.csv<\/span><\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n\n
You could also create help pages for your CLI, show progress while training (or any other task,) and meaningfully style logs and errors in the terminal window.<\/p>\n\n\n\n
Creating your first CLI in Python<\/h2>\n\n\n\n
We will be using a Python library called Typer to create CLIs. Typer is a minimal framework to create CLI commands, print stylish output, and show progress.<\/p>\n\n\n\n
You can still use Typer for creating CLIs for your non-Python programs. You have to use either a Python subprocess or communicate with your non-Python code through HTTP or a message broker. We won\u2019t discuss them here. But, it may be a topic for a future article.<\/p>\n\n\n\n
Installing Typer is straightforward. You can use PyPI:<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
pip<\/span> <\/span>install<\/span> <\/span>typer<\/span><\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n\n
Once the installation is complete, you can try this Hello World app to get a sense of how Typer works.<\/p>\n\n\n\n
Create a file called app.js in your project directory (you can pick a different name) and the below content.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
import<\/span> <\/span>typer<\/span><\/span>\n<\/span>\ndef<\/span> <\/span>say_hello<\/span>()<\/span>:<\/span><\/span>\n    <\/span>typer.secho(f<\/span>"Hello World!"<\/span>,<\/span> <\/span>fg=typer.colors.WHITE,<\/span> <\/span>bg=typer.colors.BLUE<\/span>)<\/span><\/span>\n<\/span>\nif<\/span> <\/span>__name__<\/span> <\/span>==<\/span> <\/span>"<\/span>__main__<\/span>"<\/span>:<\/span><\/span>\n    <\/span>typer.run(say_hello<\/span>)<\/span><\/span>\n<\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n\n
Running the above command will print a colorful Hello World message on the terminal<\/a>.<\/p>\n\n\n
\n
\"Running<\/figure><\/div>\n\n\n
Of course, the above exercise is more than a simple Hello World. You can change the text color and font color. If you are using an editor with IntelliSense, like VSCode, finding them is easy. If not, you can find them on Typer\u2019s documentation<\/a>, which is worth checking anyway.<\/p>\n\n\n
\n
\"CLI<\/figure><\/div>\n\n\n
Before moving on, let\u2019s also look at how to add multiple commands to your CLI. It needs a slight modification to the codebase.<\/p>\n\n\n\n
We create an instance of the Typer and call it in the \u2018main\u2019 method. It needs a slight modification to the codebase. We can use the \u2018command\u2019 decorator on functions to convert each of them into a CLI command.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
import<\/span> typer<\/span><\/span>\n<\/span>\napp <\/span>=<\/span> typer<\/span>.<\/span>Typer<\/span>()<\/span><\/span>\n<\/span>\n<\/span>\n@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>say_hello<\/span>():<\/span><\/span>\n    typer<\/span>.<\/span>secho<\/span>(<\/span>f<\/span>"Hello World!"<\/span>,<\/span> <\/span>fg<\/span>=<\/span>typer<\/span>.<\/span>colors<\/span>.<\/span>WHITE<\/span>,<\/span> <\/span>bg<\/span>=<\/span>typer<\/span>.<\/span>colors<\/span>.<\/span>BLUE<\/span>)<\/span><\/span>\n<\/span>\n<\/span>\n@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>say_hello_in_red<\/span>():<\/span><\/span>\n    typer<\/span>.<\/span>secho<\/span>(<\/span>f<\/span>"Hello World!"<\/span>,<\/span> <\/span>fg<\/span>=<\/span>typer<\/span>.<\/span>colors<\/span>.<\/span>WHITE<\/span>,<\/span> <\/span>bg<\/span>=<\/span>typer<\/span>.<\/span>colors<\/span>.<\/span>RED<\/span>)<\/span><\/span>\n<\/span>\n<\/span>\nif<\/span> __name__ <\/span>==<\/span> <\/span>"<\/span>__main__<\/span>"<\/span>:<\/span><\/span>\n    <\/span>app<\/span>()<\/span><\/span>\n<\/span><\/code><\/pre>Python<\/span><\/div>\n\n\n\n
With this new arrangement, you can have more than one command under the same CLI. You can mention the function following the filename to tell the CLI which one to execute.<\/p>\n\n\n
\n
\"Colorful<\/figure><\/div>\n\n\n
Isn\u2019t this awesome already? Now that we\u2019ve installed and tested Typer, let\u2019s move on to integrating an ML algorithm.<\/p>\n\n\n\n
Creating CLI for K-Means algorithm.<\/h2>\n\n\n\n
I\u2019ll be using the K-Means algorithm, which I have discussed in a previous post. K-Means is a simple and powerful clustering technique to group data points.<\/p>\n\n\n\n
Here is a modified version of our app. We\u2019ve created two commands; one for train and save the K-Means model and one to load and use in predictions. Note that the train function definition takes an argument, file_path. Typer will convert it into a command-line argument.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
import<\/span> typer<\/span><\/span>\n<\/span>\nimport<\/span> pandas <\/span>as<\/span> pd  <\/span># Pandas for reading data<\/span><\/span>\nfrom<\/span> sklearn<\/span>.<\/span>cluster <\/span>import<\/span> KMeans  <\/span># KMeans Clustering itself<\/span><\/span>\nfrom<\/span> joblib <\/span>import<\/span> dump<\/span>,<\/span> load  <\/span># Required to persist the trained model<\/span><\/span>\n<\/span>\napp <\/span>=<\/span> typer<\/span>.<\/span>Typer<\/span>()<\/span><\/span>\n<\/span>\n<\/span>\n@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>train<\/span>(<\/span>file_path<\/span>:<\/span> <\/span>str<\/span>):<\/span><\/span>\n    typer<\/span>.<\/span>secho<\/span>(<\/span>f<\/span>"Training K-Means with file <\/span>{<\/span>file_path<\/span>}<\/span>"<\/span>)<\/span><\/span>\n<\/span>\n    <\/span># Code for training K-Means clustering<\/span><\/span>\n    df <\/span>=<\/span> pd<\/span>.<\/span>read_csv<\/span>(<\/span>file_path<\/span>)<\/span><\/span>\n    kmeans <\/span>=<\/span> <\/span>KMeans<\/span>(<\/span>n_clusters<\/span>=<\/span>2<\/span>,<\/span> <\/span>random_state<\/span>=<\/span>0<\/span>).<\/span>fit<\/span>(<\/span>df<\/span>[[<\/span>"<\/span>Age<\/span>"<\/span>,<\/span> <\/span>"<\/span>Income<\/span>"<\/span>]])<\/span><\/span>\n<\/span>\n    typer<\/span>.<\/span>secho<\/span>(<\/span>"<\/span>CLUSTER CENTERS<\/span>"<\/span>)<\/span><\/span>\n    typer<\/span>.<\/span>echo<\/span>(<\/span>kmeans<\/span>.<\/span>cluster_centers_<\/span>)<\/span><\/span>\n<\/span>\n    <\/span>dump<\/span>(<\/span><\/span>\n        kmeans<\/span>,<\/span> <\/span>"<\/span>model.joblib<\/span>"<\/span><\/span>\n    <\/span>)<\/span>  <\/span># This line is to store the model to use in predictions later.<\/span><\/span>\n<\/span>\n<\/span>\n@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>predict<\/span>():<\/span><\/span>\n    typer<\/span>.<\/span>secho<\/span>(<\/span>"<\/span>Will be implemented soon<\/span>"<\/span>)<\/span><\/span>\n    <\/span>pass<\/span><\/span>\n<\/span>\n<\/span>\nif<\/span> __name__ <\/span>==<\/span> <\/span>"<\/span>__main__<\/span>"<\/span>:<\/span><\/span>\n    <\/span>app<\/span>()<\/span><\/span>\n<\/span><\/code><\/pre>Python<\/span><\/div>\n\n\n\n
Running the \u2018train\u2019 command of our app CLI with a file path will do the trick. You can practice it for yourself by implementing the predict command.<\/p>\n\n\n
\n
\"CLI<\/figure><\/div>\n\n\n
Showing the progress bar in your CLI<\/h2>\n\n\n\n
For consuming tasks, you\u2019ll have to show a progress bar so that the user doesn\u2019t have to pull their hair out. The Typer API helps you create progress bars without pulling yours.<\/p>\n\n\n\n
Let\u2019s put in place another command to find out the optimal number of clusters using the elbow method. This method will run K-Means many times with different numbers of groups. The ideal number would be the one with low inertia. In large applications, this might be a task running for several hours.<\/p>\n\n\n\n
Here is the code snippet that does the job. Note that, this time, we\u2019ve added two arguments and one of them has a default value. Typer will consider it as an option rather than an argument. You may choose to leave it blank.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>elbow<\/span>(<\/span>file_path<\/span>:<\/span> <\/span>str<\/span>,<\/span> <\/span>max_clusters<\/span>:<\/span>int<\/span>=<\/span>10<\/span>):<\/span><\/span>\n    errors <\/span>=<\/span> <\/span>[]<\/span>  <\/span># Create an empty list to collect inertias<\/span><\/span>\n<\/span>\n    df <\/span>=<\/span> pd<\/span>.<\/span>read_csv<\/span>(<\/span>file_path<\/span>)<\/span><\/span>\n<\/span>\n    <\/span># Loop through some desirable number of clusters<\/span><\/span>\n    <\/span># The KMeans algorithm's fit method returns a property called inertia_ that has the information we need<\/span><\/span>\n    <\/span>for<\/span> k <\/span>in<\/span> <\/span>range<\/span>(<\/span>2<\/span>,<\/span> max_clusters<\/span>):<\/span><\/span>\n        time<\/span>.<\/span>sleep<\/span>(<\/span>1<\/span>)<\/span><\/span>\n        errors<\/span>.<\/span>append<\/span>(<\/span><\/span>\n            <\/span>{<\/span><\/span>\n                <\/span>"<\/span>inertia<\/span>"<\/span>:<\/span> <\/span>KMeans<\/span>(<\/span>n_clusters<\/span>=<\/span>k<\/span>,<\/span> <\/span>random_state<\/span>=<\/span>0<\/span>)<\/span><\/span>\n                <\/span>.<\/span>fit<\/span>(<\/span>df<\/span>[[<\/span>"<\/span>Age<\/span>"<\/span>,<\/span> <\/span>"<\/span>Income<\/span>"<\/span>,<\/span> <\/span>"<\/span>Debt<\/span>"<\/span>]])<\/span><\/span>\n                <\/span>.<\/span>inertia_<\/span>,<\/span><\/span>\n                <\/span>"<\/span>num_clusters<\/span>"<\/span>:<\/span> k<\/span>,<\/span><\/span>\n            <\/span>}<\/span><\/span>\n        <\/span>)<\/span><\/span>\n<\/span>\n    <\/span># for convenience convert the list to a pandas dataframe<\/span><\/span>\n    df_inertia <\/span>=<\/span> pd<\/span>.<\/span>DataFrame<\/span>(<\/span>errors<\/span>)<\/span><\/span>\n<\/span>\n    typer<\/span>.<\/span>echo<\/span>(<\/span>df_inertia<\/span>)<\/span><\/span>\n<\/span><\/code><\/pre>Python<\/span><\/div>\n\n\n\n
Now try running python app.py elbow voters_demo_sample.csv.<\/p>\n\n\n
\n
\"CLI<\/figure><\/div>\n\n\n
This code snippet currently doesn\u2019t have a progress bar. Running this will keep the terminal without any output for several seconds and print it all at once.<\/p>\n\n\n\n
Let\u2019s go ahead and put a progress bar to help our users. Here is the updated script. Note the slight change in the for-loop.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>elbow<\/span>(<\/span>file_path<\/span>:<\/span> <\/span>str<\/span>,<\/span> <\/span>max_clusters<\/span>:<\/span> <\/span>int<\/span> <\/span>=<\/span> <\/span>10<\/span>):<\/span><\/span>\n    errors <\/span>=<\/span> <\/span>[]<\/span>  <\/span># Create an empty list to collect inertias<\/span><\/span>\n<\/span>\n    df <\/span>=<\/span> pd<\/span>.<\/span>read_csv<\/span>(<\/span>file_path<\/span>)<\/span><\/span>\n<\/span>\n    <\/span># Loop through some desirable number of clusters<\/span><\/span>\n    <\/span># The KMeans algorithm's fit method returns a property called inertia_ that has the information we need<\/span><\/span>\n    <\/span>with<\/span> typer<\/span>.<\/span>progressbar<\/span>(<\/span>range<\/span>(<\/span>2<\/span>,<\/span> max_clusters<\/span>))<\/span> <\/span>as<\/span> progress<\/span>:<\/span><\/span>\n        <\/span>for<\/span> k <\/span>in<\/span> progress<\/span>:<\/span><\/span>\n            time<\/span>.<\/span>sleep<\/span>(<\/span>1<\/span>)<\/span><\/span>\n            errors<\/span>.<\/span>append<\/span>(<\/span><\/span>\n                <\/span>{<\/span><\/span>\n                    <\/span>"<\/span>inertia<\/span>"<\/span>:<\/span> <\/span>KMeans<\/span>(<\/span>n_clusters<\/span>=<\/span>k<\/span>,<\/span> <\/span>random_state<\/span>=<\/span>0<\/span>)<\/span><\/span>\n                    <\/span>.<\/span>fit<\/span>(<\/span>df<\/span>[[<\/span>"<\/span>Age<\/span>"<\/span>,<\/span> <\/span>"<\/span>Income<\/span>"<\/span>,<\/span> <\/span>"<\/span>Debt<\/span>"<\/span>]])<\/span><\/span>\n                    <\/span>.<\/span>inertia_<\/span>,<\/span><\/span>\n                    <\/span>"<\/span>num_clusters<\/span>"<\/span>:<\/span> k<\/span>,<\/span><\/span>\n                <\/span>}<\/span><\/span>\n            <\/span>)<\/span><\/span>\n<\/span>\n    <\/span># for convenience convert the list to a pandas dataframe<\/span><\/span>\n    df_inertia <\/span>=<\/span> pd<\/span>.<\/span>DataFrame<\/span>(<\/span>errors<\/span>)<\/span><\/span>\n<\/span>\n    typer<\/span>.<\/span>echo<\/span>(<\/span>df_inertia<\/span>)<\/span><\/span>\n<\/span><\/code><\/pre>Python<\/span><\/div>\n\n\n\n
Here\u2019s what running this now would look like:<\/p>\n\n\n
\n
\"CLIs<\/figure><\/div>\n\n\n
Creating a man page for your CLI.<\/h2>\n\n\n\n
Man pages are documentation to help the user. Typer is smart to convert your functions and their inputs into detailed man pages. It\u2019ll list out all the available commands, arguments, and options.<\/p>\n\n\n\n
You can access the Typer-generated man pages with the suffix – -help.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
python<\/span> <\/span>app.py<\/span> <\/span>--help<\/span><\/span>\n<\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n
\n
\"Document<\/figure><\/div>\n\n\n
You can access the man page of a specific command as well:<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
python<\/span> <\/span>app.py<\/span> <\/span>elbow<\/span> <\/span>--help<\/span><\/span><\/code><\/pre>Bash<\/span><\/div>\n\n\n
\n
\"Typer<\/figure><\/div>\n\n\n
If you need to give the user more information, you can use a multi-line comment at the beginning of the function. Typer will display them on the help page.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
@<\/span>app<\/span>.<\/span>command<\/span>()<\/span><\/span>\ndef<\/span> <\/span>elbow<\/span>(<\/span>file_path<\/span>:<\/span> <\/span>str<\/span>,<\/span> <\/span>max_clusters<\/span>:<\/span> <\/span>int<\/span> <\/span>=<\/span> <\/span>10<\/span>):<\/span><\/span>\n    <\/span>"""<\/span><\/span>\n    This command will run K-Means algorithm many times and print the inertia values to the console.<\/span><\/span>\n    <\/span>"""<\/span><\/span>\n    ...<\/span><\/span>\n<\/span><\/code><\/pre>Python<\/span><\/div>\n\n\n
\n
\"Tyeper<\/figure><\/div>\n\n\n
Conclusion<\/h2>\n\n\n\n
In this article, we created a CLI to aid interaction with machine learning<\/a> models. You can extend these techniques beyond data science projects<\/a>. A CLI could handle any user interaction.<\/p>\n\n\n\n
Although we\u2019ve used Python to generate CLIs, you may use it to run other programs as well. In such cases, you may have to use either subprocess, HTTP, or a message broker.<\/p>\n\n\n\n
CLIs are a quick fix for a crucial problem. You can build one if developing a web portal or other solutions aren\u2019t a good fit. Also, you don\u2019t have to worry about unforeseeable events like server downtime. It makes CLIs a robust alternative to consider.<\/p>\n\n\n\n
With Typer, a minimalistic Python library, we created<\/p>\n\n\n\n