Debug Python Scripts Like a Pro

March 20, 2022

Debug Python Scripts Like a Pro

It took me some time to grasp the idea of debugging.

I'm sure that's common with most code newbies. To me, as a self-taught Python programmer, locating issues and fixing them in the quickest possible way was challenging.

But over the years, I learned several techniques to spot bugs in my scripts. Some had changed, and some I discontinued. I'll take you through what I consider the bad, the lovely, and the smart ways of debugging a Python code in this post.

Related: You Are Not Still Using Virtualenv, Are You?

Let's begin with the bad.

Print statements

Perhaps 'print' is the most used Python function of all.

Naturally, anyone who wants to know what's happening in their code uses print statements. Because that's the easiest and the most widely available choice. And it seems to work fine for smaller scripts too.

Go ahead and use it if you intend to test whether your code reaches a point or to check a variable's value at a specific point. And also if it's only for one time.

Most people use print statements to debug Python scripts. It's often the easiest way to inspect variable values at various points as well as test if the code reaches a specific point.

But as your code grow bigger, you're in trouble. Print statements are going to be on all over the project. You'll quickly lose track of which printed line corresponds to your code's print statement.

Although print statements are the easiest technique in debugging, it makes the output less understandable.

It gets more complicated if your project executes codes parallel or asynchronous. And it's not uncommon these days to use parallel execution either.

An immediate solution to this problem is to use a description in every print statement.

One way to make outputs meaningful is to print values along with a description. It makes debugging a bit more easy.

But this is going to suck the soul out of a developer. There's a better way of doing this. That's our following technique on the list.

Related: 5 Python GUI Frameworks to Create Desktop, Web, and Even Mobile Apps.

Icecream

Print statements are a tedious, time-consuming process. Icecreams make debugging sweet.

Icecream is a Python package that lets you inspect variables at different points in your script. In a nutshell, it prints the value of variables, the function executed, and the line number and file names in some instances.

Here's how to use Icecream to debug your Python code.

Step I : Install Icecream using pip.

pip install icecream

Step II : Import Icecream to your Python script

from icecream import ic

Step III: Wrap your function calls and statements with ic

Icecream is a Python package and a better alternative for regular print statements to debug Python scripts. It prints the value along with it's contextual information. In larger application, the developer don't have to think about using descriptions with print statements all the time.

Icecream makes the life of a developer easy by automatically adding the context to every log it creates. That's a lovely way to debug Python code.

But this still is additional work.

Modern IDE like PyCharm and VSCode allows you to stop the code at a specific line and inspect all the local and global variables.

Here's the smart way of debugging.

Related: This is How I Create Dazzling Dashboards Purely in Python.

How to debug Python code in a modern IDE

Live-debugging is a way to speak with your code when running.

Some of us started coding in notepad apps. Later we switched to Notepad++. Then came the revolutionary Sublime Text and Atom editors. They are great tools of their times.

Some of us started with PyCharm. I was one of them who fell in love with this excellent IDE. Yet, I switched to VSCode, because it's free, and its community was (and still is) skyrocketing.

Not to my surprise, debugging with VSCode is super easy for even beginners. That's what the remainder of this post is about. Let's break this section into three: debugging with breakpoints, configuration for popular frameworks, and debugging data frames.

Related: How to Create Stunning Web Apps for your Data Science Projects

Related: How to Create Progressive Web Apps (PWA) in Python?

Debugging Python scripts with breakpoints in VSCode.

You can start a debugger in just three steps. Here's how:

VSCode's live debugging toolkit is fantastic to temporarily pause the code at any line and inspect the variable values at that point. Basic debugging only requires three steps. 1) Click on debug icon on the sidebar 2) Set breakpoints and 3) Start the debugger.

  1. Click on the debugger on the sidebar. It's this play button with a bug on it.
  2. Create breakpoints in your code. You can do it by clicking before the line number. A red indicator will appear for every breakpoint you create.
  3. Now, start the debugger by clicking the "Run and Debug" button and selecting "Python file" in the dropdown.

As you do this, VSCode will spin up a thread to run your script and stop where you have your first breakpoint. The screen will appear as follows:

VSCode pauses at breakpoints. A debugger toolbar appears at the top to control the interpreter. You can even move the interpreter line by line. The variable section in the sidebar will display the values of local and global variable at the current line where the interpreter is.

As it stopped in the breakpoint you created, you can now see the line is highlighted, the sidebar has changed, and a new toolbar appears on the top right (this location may differ in your window) corner.

The toolbar is the controller for your debugger. The play button will ignore the current breakpoint and move on to the next. The stop button will stop the debugger, and the refresh button will restart the debugger all over again.

I find the step-in button extremely useful. It's the down arrow key on the debugger toolbar. Please see the below recording to understand how it works and how useful it is.

VSCode debugger allows us to execute a script line by line. The arrow-down and arrow up buttons on the debugger toolbar helps controlling this. The variable pannel will update as per the current line being executed.

My first breakpoint was on line number 14. This line calls a function called 'sum_odd_numbers.' As I click the step in key, the interpreter moves to the next line but stops there. It is within the function I call (line number 4.)

Pay attention to the variables section in the sidebar. As we are now within the function, the scope changes. Under the 'local' area, we see the value of n is 100. The value of v is available because the interpreter hasn't executed line number 4 yet.

But as you keep clicking on the step-in key. You see, the value of 'v' is changing as well. As we step inside the loop, you can also see the value of 'i'.

Notice that we get to line number 8 only when the value of 'i' is odd. As we execute line number 8, the value of 'v' increases by the value of 'i' at that time.

Great, we could narrate everything line by line while our script was running. That's why live debugging is the more brilliant way to find and fix issues on your code.

Configuring VSCode to debug popular Python frameworks.

VSCode supports popular frameworks such as Django, Flask, FastAPI, and Pyramid out of the box. In most cases, you don't have to configure anything special for these frameworks. You only have to select the framework in the dropdown when starting the debugger.

If this is the first time you're working on a Django project, you can install django with pip install django and create a django project using the command Django startproject <project_name>

But in certain conditions, you will have to override the defaults. For instance, if another process blocks the 8000 port, Django needs a different one to start. You can give additional information to the debugger using a launch.json file.

VSCode debugger supports popular Python frameworks such as Django, FastAPI, Flask and Pyramid out of the box. You can simply select the debugger and in the dropdown that appears, choose your the one you want to debug.

Go to the debugger window and select the "create a launch.json file" option. In the dropdown, choose the framework you're using. I've chosen Django for this illustration.

You'll see a new JSON file open up with some familiar options.

You can override the default debug configurations on the launch.json file. You can specify arguments to the command in a list and more configuration options are available through VSCode&#x27;s auto suggestion.

The above file runs the 'manage.py' file in the ${workspacefolder} (that's the root of the project you've opened in VSCode). In addition, it takes a command-line argument 'runserver.'

What it means is it's about to execute python manage.py runserver when the debugger starts.

I've added another argument by extending the 'args' list. In line number 11, I configured the Django dev server's port to be 5000 instead of its default 8000. This modification will let the debugger run python manage.py runserver 5000 .

Go to a new line and press the space holding your ctrl (or command) key to see what other options you have. As shown in the figure above, a list of options will pop up.

Let's see the debugger in action.

Replace the content of /urls.py with the following content.

from django.urls import path

from views import sum_odd_numbers

urlpatterns = [
    path("sum_odd_numbers/<int:n>", sum_odd_numbers),
]

And create a file called views.py with the following content. We're running the same odd number counter as a web service.

from django.http import HttpResponse
from django.http.request import HttpRequest

def sum_odd_numbers(request: HttpRequest, n: int) -> HttpResponse:
    """Sum all the odd numbers less than n"""

    v = 0

    for i in range(n):
        if i % 2 == 1:
        v += i

    return HttpResponse(v)

Go ahead and start the debugger by clicking the play button in the sidebar. Now, in your web browser, if you go to http://127.0.0.1:5000/sum_odd_numbers/10, the browser will show a result of 25. Try a different number instead of 10 in the URL to see how the webserver works.

The output of sample web response on the browser

Now, Let's put a breakpoint at line 7 and revisit the URL on your browser again. This time it the browser keeps waiting for the response. But VSCode stopped processing the request at our breakpoint.

It&#x27;s easy to inspect variables in a Django web request using VSCode&#x27;s debugger tool. Set a break point at any line in your Django view. and call the URL in the usual way. It&#x27;ll stop at the breakpoint and you are able to access the request object. At this point django request object will even have the logged in user if the request is authenticated.

Look at what you've got in the variables panel. You can access your entire request object, including information regarding the path, type of request, and browser cookie information.

It's effortless to debug web servers in live debugging mode. We'll discuss one of the most recent features that let me debug data frames in the next part.

Debugging pandas dataframes with VSCode

VSCode has already been very useful in data science with an in-built Jupyter notebook in it.

Yet, debugging an analysis project had been difficult if it was on a script. A very common encounter in my day-to-day work is processing datasets inside web requests.

Lists, dictionaries, and even objects are relatively straightforward to show in the variables section. But a pandas data frame is a complex object that needs to be in a tabular format. Only then does it make sense for the developer to find issues in it.

VSCode has solved this issue in the best possible way. You can simply right-click the data frame object in the variables section and choose 'View Value in Data Viewer.'

VSCode is very helpful in debugging data science and machine learning projects. Most data science projects includes pandas data frames. You can put a breakpoint at any line and when the executor stops at that point you can inspect the data frame. By right-clicking and opening it on the Data Viewer, you can even see it in a tabular view.

In the above screenshot, I've added a breakpoint at line number 5. At this time, the interpreter has read the CSV and the data frame is in the memory. When we open it in the Dat Viewer, we get the following.

Debugging pandas dataframes is easy with VSCode's Data Viewer. It's a new utility available in the newer version of VSCode's Python extension. You can put a breakpoint at any line in the code and the debugger will pause at that point when it runs. You can inspect any data frames available at that time by right clicking on it's reference in the variable section of the debugger sidebar. It'll open up a tabular data frame and you can even filter it.

It's a regex-supported filterable tabular view of our data frame. This feature is a handy technique to debug projects when you aren't on a Jupyter notebook. That's the case when finding and fixing bugs in most production systems.

If you don't see the data viewer option on the right-click menu, you have to install or update the Python extension in VSCode. Only the recent versions have this feature.

Final thoughts

Debugging can be very challenging, regardless of how experienced you are.

For most beginners, print statements are the easiest to do. But having print statements to find bugs in large production-grade systems is not practical. Only the right tools and techniques can help ease the work.

A lovely tool to rescue is Icecream. Icecream automatically includes a description of the context in which it logs the statement. Yet, every time you want to test something different, you'll have to rerun the entire script.

Live debugging is the smartest way to find and fix issues in software projects. Most modern IDEs support live debugging. In this post, we covered how the VSCode debugger works.

I hope this guide helps make your code bug-free.

Thanks for the read, friend. It seems you and I have lots of common interests. Say Hi to me on LinkedIn, Twitter, and Medium. I’ll break the ice for you.
Not a Medium member yet? Please use this link to become a member because I earn a commission for referring at no extra cost for you.

How we work

Readers support The Analytics Club. We earn through display ads. Also, when you buy something we recommend, we may get an affiliate commission. But it never affects your price or what we pick.

Connect with us