How to Build a Python App That Monitors Amazon Prices

Automate the boring stuff.

I buy coffee on Amazon. My current favorite is Three Sisters by Kicking Horse Coffee. (Bonus points if you know which Three Sisters I’m talking about.)

One thing that I’ve noticed is that the price is always changing. I’ve seen it as low as $7 a bag, and upwards of $11. It got to the point where I was checking the price daily, and I would buy a couple of bags when I saw the price under $8.

Today we’re going to write some software to automate this process. We’ll create a python web scraper that periodically checks the price, and sends me a text message when the price drops below a certain threshold.

Let’s get started.

Create our Environment

The first thing we’ll do is set up our python environment and install dependencies. I’ll be using conda, but feel free to use venv.

We’ll create a folder called amazon-monitor and call our environment amzn.

$ mkdir amazon-monitor && cd amazon-monitor
$ conda create --name amzn python=3.7
$ conda activate amzn

Next we’ll install our dependencies: requests and BeautifulSoup. We’ll also export our dependencies to environment.yml (requirements.txt if you’re using venv).

$ conda install -c anaconda requests beautifulsoup4
$ conda env export > environment.yml

Create the Amazon Scraper

Create a file called amazon-monitor.py and open it in your favorite code editor. I use Atom.

Import our dependencies:

# Imports
import requests
from bs4 import BeautifulSoup

Next we’ll create a function that grabs the product page from Amazon and returns the price.

def get_price(url):
    # Set User-Agent header so Amazon returns the actual page
    headers = {
      'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'
      }

    # Get the page
    response = requests.get(url, headers=headers)

    # Load into BeautifulSoup
    soup = BeautifulSoup(response.content, 'html.parser')
    # A trick since Amazon's page is javascript heavy
    soup = BeautifulSoup(soup.prettify(), 'html.parser')

    # Get the price
    price = soup.find(id='priceblock_ourprice').get_text()

    # Strip off the '$' and parse as a float
    price = float(price[1:])

    return price

Okay, let’s go over this code.

The first thing we have to do is set the User-Agent header, otherwise Amazon won’t return the product page. I just grabbed a generic User-Agent string.

Next we call the get() function in the requests library. This is the function that actually calls the URL and returns the HTML page.

We then load the response’s content into BeautifulSoup. We actually do this twice, otherwise the code doesn’t work properly. It’s a little trick since the page returned by Amazon is javascript-heavy.

Back in the browser, if we right click on the price and select Inspect, we can see that the price has an ID of “priceblock_ourprice”. We will use this to select the price in BeautifulSoup.

We now have the price as a string. We want to convert this into a number. So we have to get rid of the ”$” sign. This is easily accomplished with python array slicing. The [1:] creates a substring of price starting at the first index, removing the ”$” at the zero index. float() parses the resulting string as a number. This is returned by the function.

Let’s Run the Code

Sanity check time. Let’s run the code and verify it’s getting the price.

if __name__ == '__main__':
    url = 'https://www.amazon.com/gp/product/B00KC0LLFQ/'

    price = get_price(url)
    print(price)

Running the code and the price will be printed to the terminal.

$ python amazon-monitor.py
10.52

The program prints out 10.52, which if you look at the screen shot above, is the correct price.

Cool cool.

Now let’s have some fun. If the price drops below $8, the program will send me a text message.

Send SMS Alert with Twilio

We’ll use Twilio to send text messages. Sign up for a free account here. Sign up and get your Twilio phone number that you’ll send texts from.

Note that free accounts can only send texts to your verified phone number, and prepends the message with “Sent from your Twilio trial account”.

Install the python Twilio library in your terminal and update the environment.yml file.

$ pip install twilio
$ conda env export > environment.yml

The Twilio API requires an account_sid and auth_token, which are found on your Twilio console. These values should be kept secret and never written directly in your source code. Store them as environment variables, and import them into your code.

First, add our new imports.

import os
from twilio.rest import Client

Next we’ll create a function called send_sms(price, title) and pass in the price and a title. Read the account_sid and auth_token from the environment variables to be used with the Twilio API. Also store your verified phone number you’ll be sending texts to, and your Twilio phone number as environment variables.

def send_sms(price, title):
    # Get the account_sid and auth_token from environment variables
    account_sid = os.getenv('ACCOUNT_SID')
    auth_token = os.getenv('AUTH_TOKEN')

    # Get the to and from phone numbers
    to_phone = os.getenv('TO_PHONE')
    from_phone = os.getenv('FROM_PHONE')

    # Send the text
    client = Client(account_sid, auth_token)
    message = client.messages.create(
        to=to_phone,
        from_=from_phone,
        body=f'{title} is currently ${price}.'
    )

    return message.sid

Sending a text message with the Twilio library is incredibly easy! Just create a Client, and then call messages.create() on the client. If the message was sent, a unique string identifier (sid) is created. We’ll return this value.

And that’s it! To test this out, I set the threshold value above the current price. Will I receive a text message?

Success!

Deploy

I may deploy this code as an AWS Lambda function one day, but right now I’m content with running this code as a cron job on my Raspberry Pi.