uOttaHack4 Challenge: Leveraging Solace PubSub+ to Build a Price Comparison Application

uOttaHack is Ottawa’s largest hackathon that takes place every year at Ottawa University. This year’s theme involved solving issues related to the pandemic. Over 300 students from a variety of backgrounds registered! As a prime sponsor of the uOttaHack4 event, Solace challenged students to leverage the PubSub+ Event Broker: Cloud technology to create something truly unique.

A student from the University of Waterloo built an app to give people a better online shopping experience during the COVID-19 pandemic.

Anees Aissaoui describes his project in detail below:

Finding the Best Online Price with the Buy It App

Due to the COVID-19 pandemic, we’re all stuck at home. If you’re anything like me, the lockdown has changed my online shopping habits quite a bit, and I find myself shopping on websites like Amazon way more than I used to.

With this change to my lifestyle, I realized that e-commerce is weirdly inconvenient when you really want to get the best price. You have to go check each retailer’s website individually and compare prices manually. I then thought to myself, “Hey, it would be really cool if I had a price comparison tool to search for the best price across a variety of online retailers.” That’s where I got the idea for Buy It, an application that does just that.

Given a product name to search, Buy It will return search results from different online retailers giving you all the information you need to save money, all in one place.

How It Works

Buy It was developed in Android Studio using Java and Kotlin. The pricing data presented in the app is actually collected via web scraping. To do that, I leveraged the ParseHub API after training their scraping bot to search e-commerce websites with whatever product the user is interested in. The Python code snippet below shows the API request for the pricing data that was used. The image is the ParseHub app where I trained the web scraping on CamelCamelCamel, an Amazon web viewer with a UI that is easier to scrape.

```

if r3['data_ready'] == True:

time.sleep(2)

r2 = requests.get('https://www.parsehub.com/api/v2/runs/' + run_token + '/data', params=params2)

```

`client.publish(f'iot/{username}/' + itemname, payload=json.dumps(r2.json()))`

The main issue is that the API isn’t easily compatible with Java, so I thought to host an API endpoint with Flask in Python to make data available for my Java app to request. But I had a better idea: using Solace’s PubSub+ Event Broker. This made a lot of sense, since an event-driven architecture solution allows for easy two-way communication between both the client and the publisher. I used MQTT, an open standard messaging protocol, to send messages from my Python client to the Solace broker on a predefined topic (f'iot/{username}/' + itemname). I then developed a Kotlin Java application to connect to the same broker using MQTT and subscribe to the same predefined topic. The code snippet below shows the app publishing the user’s item search to a topic named “iot/cranky-schtern/product”. The Python client is subscribed to this topic and will receive this message. After that, the app subscribes to the topic named after the user’s item search.

```

mqttClient.publish("iot/cranky-shtern/product", editTextMsgPayload.<em>text</em>.toString())

mqttClient.subscribe("iot/cranky-shtern/" + editTextMsgPayload.<em>text</em>.toString())

```

Of course, for the Python client to receive the message published to the product topic, it has to subscribe to it too.

```

def on_connect(client, userdata, flags, rc):

print(f'Connected (Result: {rc})')

client.subscribe(f'iot/{username}/product')

```

Later, the Kotlin code in the app will receive the JSON payload of the message published from the Python client, as you can see below:

```

override fun messageArrived(topic: String, mqttMessage: MqttMessage) {

Log.w("Debug", "Message received from host '$<em>SOLACE_MQTT_HOST</em>': $mqttMessage")

textViewNumMsgs.<em>text </em>= ("${textViewNumMsgs.<em>text</em>.toString().<em>toInt</em>() + 1}")

val str: String = "------------"+ Calendar.getInstance().<em>time </em>+"-------------\n$mqttMessage\n${textViewMsgPayload.<em>text</em>}"

textViewMsgPayload.<em>text </em>= str



val jsonResponse = mqttMessage.toString()

}

```

Using this architectural paradigm with a message broker, I was able to abstract the messaging layer allowing clients of different languages to communicate with each other and pass data. The Python client connected to the message broker using a messaging protocol and so did the Java client. The advantages to using PubSub+ Event Broker was the multi-language and multi-protocol support, meaning that I can have another client written in another language using another supported messaging protocol and still be able to receive the same message. This is extremely powerful as it decouples the dependency on a specific language or protocol-oriented technology stack for my overall solution.

That means my app can send the user’s product search to the Python script, Python runs the API on that search to initiate a web scraping action, then the Python application publishes  whatever data it collects to PubSub+ Event Broker. Meanwhile, the Java app was simply waiting for messages on the topic of the search product. There was no need to constantly poll for updates from a REST endpoint  ̶  it was an “event-driven” application! After receiving a response, the Java app can then process the pricing information and display it to the user.

Another cool thing about using Solace PubSub+ Event Broker is that it’s an extremely scalable solution. I could make this app track prices real time from a large number of retailers at once. In other words, I’d be generating constant heavy network traffic, and the event broker would handle any needed back pressure. By leveraging PubSub+, I was able to make a quick, easy, and powerful communication pathway for all the components of my hack to talk through, while also future-proofing my app for further development and updates.

What I Learned

I’m super excited about how much I’ve learned. I used Solace’s event broker to replace a REST API endpoint, which was useful and educational. Understanding the interactions between different elements in a project is very valuable.

I’ve also never built a mobile app, never done web scraping, never coded in Python or Java/Kotlin, so lots of new stuff. I’m constantly impressed at the number of things you can learn in a single project.

Authors