Building a Basic Website Monitoring Bot with Python — Part 1

Photo by NeONBRAND on Unsplash

Hello guys 👋, welcome to another python tutorial. Today we will be building a basic website monitoring bot that logs the status of a website. Monitoring is an essential part of DevOps as it enables you to gain insight into the state of your application and the underlying infrastructure. The bot we will be building today will just continuously ping a website URL and log the result, whether it is online or not. Later, we will direct these logs from the console to a text file and also write code to send us an email whenever the status of our application changes. Without further ado, let’s get into it.

Note: This tutorial assumes you have at least a basic knowledge of python

Requirements

For this tutorial, we will need just a few python modules.

1. Python….obviously
2. Requests
3. smtplib
4. time

You need to have python installed. Install it here. We will be using requests to send HTTP requests to the server and smtplib is our email client. You can install the requests modules with python3 -m pip install requests smtplib however, comes inbuilt with python. The time module provides helpful functions relating to date and time. It comes inbuilt with python too.

Sending requests

So to get started, create a file called monitoring-bot.py in any directory of your choice and open it with your favourite code editor or IDE.

First of all, let’s import the libraries we need. Let’s start with requests and time. Add this code at the top of the file

import requests, time

We aren’t importing smtplib as we do not need it for now. Let us create a function that pings the website and returns if it is online or not. the ping function requires a URL parameter which we will set as a global parameter.

def ping(url):
    pass

We will fill in the code later. Below that, let us create an infinite loop that continuously calls our ping function and logs info to the console.

url = "http://localhost"
while True:
    try:
        isOnline = ping(url)
        print(isOnline)
    except:
        print("Couldn't connect to website")
    time.sleep(2)

Ok, so firstly, I set the url parameter to a website I have running on localhost port 80. You can set this to whatever you want. Next, I call the ping function and pass in the url and set the isOnline variable to the result of the function call. Next I print out the isOnline variable which is a True or False value telling us whether the website is online or not. Notice that all this code is in a try and except block. This is to prevent the script from exiting unexpectedly if we can’t connect to the server. We then call the time.sleep function and pass in the number of seconds we want to sleep for which in this case is two. This stops the execution of our program for two seconds to prevent flooding our website with requests.

Now let’s flesh out the ping function.

def ping(url):
    result = requests.get(url)
    return result.status_code == 200

We send a get request to the url and return whether or not the status code is equal to 200. A HTTP status code of 200 means that the request was completed successfully. Here’s a link to read more on status codes.

Ok time to test out what we've done.

As you can see, the website isn't up so we get a couldn’t connect to website message. Ok, let us start up an nginx docker container running on port 80 so our bot can send requests to it.

Now our bot is printing out True. Let us stop the docker container and see what happens.

We get the “Couldn’t connect to website” message again. Great, the bot is working. However, we want to show some meaningful information on the console so replace the code block in the while loop with this

while True:
    try:
        isOnline, method = ping(url)
        if(isOnline == True):
            print(f"[{time.strftime('%D')},   {time.strftime('%H:%M:%S')}] {method} {url} The website is online")
        else:
            print(f"[{time.strftime('%D')}, {time.strftime('%H:%M:%S')}] {method} {url} The website is offline")
    except:
        print(f"[{time.strftime('%D')}, {time.strftime('%H:%M:%S')}] Couldn't connect to website")
    time.sleep(2)

That must be a lot to take in! First of all, our ping method now returns the HTTP method of the request. Read more about HTTP methods here. We are also checking if the website is online or not and printing out different values accordingly. In our print statement, we are making use of f-string, read more about f-strings here, which allow us to format strings inline by with the use of the curly braces, {}. In the strings, we show the date and time the request was made, the request HTTP method, the url and finally a message telling us the status of the website. We get the current date and time by calling the time.strftime function which accepts a string of formatting options and returns the date/time formatted accordingly. The formatting options we use here are:

1. “%D” — This formatting option returns the current day
2. “%H” — This returns the current hour
3. “%M” — This returns the current minute
4. “%S” — This returns the current second

Read more about the time module and the time.strftime function here. Ok, now lets modify the ping method to return the request method as well

def ping(url):
    result = requests.get(url)
    method = result.request.method
    return (result.status_code == 200, method)

Ok, let’s run this and see the output

As you can see, it works. Let’s stop the website and see the result

Ok, everything seems to be working. However, in a production environment, you would likely want to log to a log file and not to the console. Let’s create a method that writes to a log file instead of the console

def log(message):
    with open("logs", "a") as logs:
        logs.write(message + "\n")

Here we are opening a file called logs.txt in append mode. This means that any new content we write will be added to the end of the file instead of over-writing the file. opening the file using the “with” operator ensures that the file is closed after all code in the “with” block has been executed. We add a “\n” to the message to automatically add a new line to each message instead of joining everything together which is the default behaviour.

Now let us replace all our print statements with the log function

while True:
    try:
        isOnline, method = ping(url)
        if(isOnline == True):
            log(f"[{time.strftime('%D')},   {time.strftime('%H:%M:%S')}] {method} {url} The website is online")
        else:
            log(f"[{time.strftime('%D')}, {time.strftime('%H:%M:%S')}] {method} {url} The website is online")
    except:
        log(f"[{time.strftime('%D')}, {time.strftime('%H:%M:%S')}] Couldn't connect to website")
    time.sleep(2)

As you can see, we are now logging to a text file. Ok, I guess that will be all for part 1 of this tutorial. Join me in part 2 where we modify the code to only log whenever the status of our website changes. We will also be implementing our email sending functionality to notify us whenever our website goes down or comes up. In part 3, we will be integrating slack so we can log to a slack channel. Bye for now👋

Ok, so I guess that will be all for now, but watch this space for more DevOps-related posts. You can reach out to me on Github or send me an email at utibeabasiumanah6@gmail.com. Thanks for reading✌️

Please leave a comment if you have any thoughts about the topic — I am open to learning and knowledge explorations.