The Reusable Python Logging Template For All Your Python Apps

Don't use print() or the default root logger, set up a project level logging like a pro

Nov 08, 2022

brown wood logs — Photo by Meritt Thomas on Unsplash

The perfect way to debug and follow the execution of an app is through well defined, informative and conveniently structured logs.

They are the essential component of any small, medium or large project in any programming language, not just Python.

I started using the Python logging library a couple years ago, and since then, I’ve been consulting countless tutorials and articles online on how to use it effectively and with best possible setup for my projects.

All of them are good at explaining how to set a logging system for a single Python script. However, it is nearly impossible to find one that explains how to set up the Python logging library to be used application-wide, and how to properly integrate and share the logging info comfortably in all your project modules.

In this article, I will be sharing my personal logging template that you can easily use for any project with multiple modules.

I am assuming you know the basics of logging already. Like I said, there are a number of good articles out there to learn from.

Okay, let’s get down to it! 👇

Making a simple Python project

Explaining a new concept should always be done at first in simpler terms with little focus diverted towards background information. With that in mind, let’s go ahead and initialise a simple project for now, shall we?

Create a folder called ‘MyAwesomeProject’. Inside it, create a new Python file called app.py. This will be the starting point of our app. I will be using this project to construct a simple working example of the template I’m talking about.

Go ahead and open your project in VSCode (or any editor you prefer).

Now, let’s create a new module for the app level logging setup. We’ll call it logger.

And we’re done with this part.

Creating the app level logger

This is the main part of our template. We create a new file called logger.py.

Let’s define a root logger and use it for initialising our app level logger. Time for some code!

A few imports and our app name:

import logging
import sys

APP_LOGGER_NAME = 'MyAwesomeApp'

The function we will be calling in our app.py:

def setup_applevel_logger(logger_name = APP_LOGGER_NAME, file_name=None): 
    logger = logging.getLogger(logger_name)        
    logger.setLevel(logging.DEBUG).   
    formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
    sh = logging.StreamHandler(sys.stdout)
    sh.setFormatter(formatter)
    logger.handlers.clear()
    logger.addHandler(sh)
    if file_name:
        fh = logging.FileHandler(file_name)
        fh.setFormatter(formatter)
        logger.addHandler(fh)
        return logger

We will be defining our logger with the predefined DEBUG level and use a Formatter to structure our logging messages. We then assign it to our stream handler in order to write messages to the console.

Next, we also make sure to include a file in which we can additionally store all our log messages too. This is done via the logging FileHandler.

Finally, we return the logger.

Another function is needed which will make sure our modules can call the logger when needed. Define a get_logger function.

def get_logger(module_name):    
   return logging.getLogger(APP_LOGGER_NAME).getChild(module_name)

Also, in order to use this module as a package, we can optionally make a folder called logger and put this file inside it. If we do that, we’ll also need to include a ___init__.py file in the folder and do

from .logger import *

to make sure we can import our module from the package.

Great! The main setup is done!

Setting up our module-level logging

A simple module to test our logger can be done to understand the template better. Let’s define it a simple module.py.

import logger
log = logger.get_logger(__name__)

def multiply(num1, num2): # just multiply two numbers
    log.debug("Executing multiply function.")
    return num1 * num2

Awesome! This module now has access to the logger and should display the message with the appropriate module name.

Let’s test it now!

Run our script and test the logger

Now, we construct the app.py.

import logger

log = logger.setup_applevel_logger(file_name = 'app_debug.log')

import mymodule

log.debug('Calling module function.')

mymodule.multiply(5, 2)

log.debug('Finished.')

Notice how we import our module after initialising the logger? Yes, it’s necessary.

Now, verify that your directory contains these files:

Finally, run the script simply via:

python3 app.py

You should get an output like:

And your directory structure should also change to include the new log file. Go ahead and inspect its contents!

Final words

This is how you can easily integrate a working logging setup in your projects. 😄.

It’s simple and you can easily extend it to include numerous hierarchies between different modules, capture and format exceptions with the logger, configure it in a more advanced manner using a dictConfig, and so on. The possibilities are endless!

The code repository is located here. It also contains the resources for all my other articles too. You can go ahead and ⭐️ and bookmark it!