MakeUofT 2020 Challenge: Leveraging Solace PubSub+ for Smart Waste Disposal & Collection

MakeUofT is Canada’s largest makeathon. Similar to a hackathon, it’s a place where projects come to life; a makeathon focuses on hardware-based projects where students build something from scratch through hardware and software integration.

The theme for MakeUofT 2020 was connectivity and machine learning. As a prime sponsor of the event, Solace challenged participants to make the best use of Solace PubSub+ Event Broker to stream events and information across cloud, on-premises, and IoT environments.

A group of four students worked together to develop a solution to smart city urban waste collection. Their project DetritusAI won the “Best Use of Solace PubSub+” Prize. Below, the DetritusAI team shares how they designed and developed their prize-winning project.

Best Use of Solace PubSub+ Prize:
Modern Smart City Urban Waste Collection

The issue of waste management is something that many people take for granted or view as trivial, but it has a massive effect on our health and lifestyle. In Canada, a developed country, only 9% of plastics are recycled. Unfortunately, the remaining 91% of plastics that ends up as waste in our landfills is equivalent to 2.8 million tonnes (the weight of 24 CN Towers). In developing nations, this is an even more serious issue that has profound impacts on quality of life.

Our team sought to tackle the challenges modern urban cities face with waste collection and waste disposal, hoping to optimize waste collection routes using machine learning and IoT.

DetritusAI – Smart Waste Disposal

DetritusAI is a smart waste disposal system that detects and categorizes objects as either waste or recycling to help users correctly identify and dispose of their garbage. After the computer vision model is run and the object is classified, the respective container lid (either trash or recycle) will open, prompting the user to dispose of their garbage in the proper container.

Additionally, the smart garbage cans will transmit essential information such as waste levels and location allowing for the optimization of garbage routes. DetritusAI tracks the quantity of waste that is in each container and communicates with the client-side applications used by garbage truck drivers to determine how full each container is. Based on the capacity of each garbage can and its location, DetritusAI calculates the optimal route for garbage trucks to collect garbage while minimizing distance and time, even taking into account traffic.

The Design

When users place an object near the garbage can, a time-of-flight sensor detects the object and triggers an image classification algorithm that identifies the category of the waste. An event is triggered via Solace PubSub+, which instructs the garbage can to open the appropriate lid.

We chose to use the Solace PubSub+ Platform to send data and notifications as we wanted our data to flow asynchronously in order to maintain speed and efficiency as opposed to traditional synchronous REST protocols. We also chose to take advantage of Solace’s wildcard features which we will go into more depth about in the next section.

Within the garbage cans, the time-of-flight sensors continuously determine the capacity of the bin and communicate that information via Solace PubSub+ Platform to the client-side application. Using the Google Directions API, the optimal route for garbage collection is determined by factoring in traffic, distance, and the capacity of each bin. An optimal route is displayed on the dashboard, along with turn-by-turn directions.

How We Built It

Bin Sensors

Sensors on the garbage bin detect the fullness of each bin and send the real-time data to the React website using Solace PubSub+. Each garbage bin will have two topics, ‘garbage’ and ‘recycle’. The garbage bins are differentiated by their location, adding another layer to the topics. An example topic is “DetritusAI/UofT/State/RecyclingVol”. This topic represents the recycling bin volume of the garbage can located at UofT. A code snippet is shown below:

void loop() 
{
  VL53L0X_RangingMeasurementData_t trash_measure;
 
  Serial.print("Reading trash measurement... ");
  lox_trash.rangingTest(&trash_measure, false); // pass in 'true' to get debug data printout!
  
  if (trash_measure.RangeStatus != 4)
  { // phase failures have incorrect data
  Serial.print("Distance (mm): "); Serial.println(trash_measure.RangeMilliMeter);
  }
  else
  {
  Serial.println(" out of range ");
  }

  float trash_vol = 100.0 - ((trash_measure.RangeMilliMeter / 300.0) * 100.0);
  Serial.print("Trash volume (%): "); Serial.println(trash_vol);
  char trash_vol_string[6];
  sprintf(trash_vol_string, "%f", trash_vol);
  client.publish("DetritusAI/UofT/State/TrashVol", trash_vol_string);

  if ((loopCounter % 200 == 0) && !client.connected()) {
    long now = millis();
    if (now - lastMqttReconnectAttempt > 5000) {
      lastMqttReconnectAttempt = now;
      // Attempt to reconnect
      if (mqttReconnect()) {
        lastMqttReconnectAttempt = 0;
      }
    }
  } else {
    // Client connected
    client.loop();
  }

  // Check for OTA Updates
  if (enableOTA && (loopCounter % 20 == 0)) {
    ArduinoOTA.handle();
  }

  if (loopCounter >= 1000) {
    loopCounter = 0;
  }

  loopCounter++;
  
  delay(1000);
}

In this snippet, the trash volume is read by the sensor, converted into a string, then sent to the “DetritusAI/UofT/State/TrashVol” topic. Both the website frontend and the path-finding AI subscribe to this topic and listen for real-time data.

Path Finding/Optimization

The pathfinding algorithm listens to the garbage bin data sent by the sensors and adjusts the garbage collection route based on fullness of the bins and real-time traffic. It then runs the algorithm that generates the optimized path, translates it to step-by-step instructions, then sends the instructions to the “RoutePlanning” Solace topic.

def find_shortest_path(client):
    locations = {0:'University of Toronto -  St. George Campus',1:'Yonge-Dundas Square'}
    URL = 'https://protect-us.mimecast.com/s/Q-lOCjRMl6c2m3mS7S2K8?domain=maps.googleapis.com'
    origin = 'Waste Management - Etobicoke Transfer Station'
    destination = 'Waste Management - Etobicoke Transfer Station'
    waypoints = "optimize:true"
    for idx in locations:
        # print(locations[idx])
        waypoints = waypoints + '|' + locations[idx]
    # print(waypoints)
    key = 'AIzaSyB0PKakIGKL9F4veyAeaD4mIXl28CDxJ-U'
    PARAMS = {'origin': origin, 'destination': destination, 'waypoints': waypoints, 'key': XXX}

    r = requests.get(url = URL, params = PARAMS) 
    data = r.json()
    directions = parse_json(data, origin, destination)
    client.publish('RoutePlanning', directions)
    traffic_time, severity = find_traffic(locations, URL, origin, destination)
    directions = directions + "Due to " + severity + " traffic, total trip may take " + traffic_time
    # print(directions)
    return directions

Note: The actual key is replaced with ‘XXX’ in the above code snippet to protect privacy.

The AI section also listens for object detection results. When an item is placed in front of the bin, a message will be sent from the bin through Solace PubSub+ and received by the object detection algorithm.

def on_message(client, userdata, msg):
print("Object detected")
client.item_detected = True

The object detection algorithm is executed when the flag set earlier becomes true.

client = mqtt.Client(transport='websockets')

mqtt.Client.item_detected = False

client.on_connect = on_connect
client.on_message = on_message

# Required if using TLS endpoint (mqtts, wss, ssl), remove if using plaintext
# Use Mozilla's CA bundle
client.tls_set(ca_certs=certifi.where())

# Enter your password here
client.username_pw_set('solace-cloud-client', 'XXX')

# Use the host and port from Solace Cloud without the protocol
# ex. "ssl://yoururl.messaging.solace.cloud:8883" becomes "yoururl.messaging.solace.cloud"
my_url = "mr2j0vvhki1l0v.messaging.solace.cloud"
client.connect(my_url, port=20009)
client.loop_start()
while True:
client.item_detected = False
# client.item_detected = True #REMOVE
while client.item_detected != True:
time.sleep(0.01)
# print(client.item_detected)
identify_detritus(client)
path = find_shortest_path(client)
time.sleep(8)
client.item_detected = True

Note: The actual password is replaced with ‘XXX’ in the above code snippet to protect privacy.

React Website

The React website listens for data from the sensors and displays them on the map markers. Different displays require subscription to the corresponding topics. Some code snippets are shown below:

export class MapSolaceDirections extends Component {
//populate with solace data
constructor(props){
super(props);
this.state = {
connected: false,
messages: ''
}

messaging.register(this.handleMessage.bind(this));

messaging.connectWithPromise().then(response => {
console.log("Succesfully connected to Solace Cloud.", response);
messaging.subscribe("RoutePlanning");

this.setState({
connected: true,
messages: this.state.messages
});
}).catch(error => {
console.log("Unable to establish connection with Solace Cloud, see above logs for more details.", error);
});
};</pre>
<h4><strong>Recycling Volume of UofT Trash Bin</strong></h4>
The website also listens for the optimized direction from the pathfinding part.
<pre>export class MapSolaceRecycle extends Component {
//populate with solace data
constructor(props){
super(props);
this.state = {
connected: false,
messages: ''
}

messaging.register(this.handleMessage.bind(this));

messaging.connectWithPromise().then(response => {
console.log("Succesfully connected to Solace Cloud.", response);
messaging.subscribe("DetritusAI/UofT/State/RecyclingVol");

this.setState({
connected: true,
messages: this.state.messages
});
}).catch(error => {
console.log("Unable to establish connection with Solace Cloud, see above logs for more details.", error);
});
};

Topic Layout and Branching

The middleware communications for this project were done using the Solace PubSub+ Platform. As IoT is inherently event-driven and extremely lightweight, we decided that a message streaming platform such as Solace PubSub+ would be the best way to implement our garbage collection system.

As opposed to traditional REST protocols, Solace PubSub+ is asynchronous and enables decoupling between applications – that is, the frontend client does not need to be constantly requesting data from the IoT devices. Rather, our devices can publish to an open Solace topic over MQTT and have the frontend UI client subscribe to the same topic. This way, we are getting a constant stream of data without needing to wait for polled synchronous requests and responses. Additionally, based on the topic hierarchy that we chose to implement, our team was able to utilize Solace’s wildcard feature, where we were able to query several topics at once to extract data efficiently.

Featured below is the structural diagram of the topics in DetritusAI.

Under the main topic (“DetritusAI”), subtopics represent locations of the bins – such as “UofT” and “1Dundas”.

There is a “State” subtopic under each location, which branches into two more subtopics “RecyclingVol” and “TrashVol“. “RecyclingVol” and “TrashVol“ represent the fullness of the corresponding bins of that specific trash can. By wildcarding the lowest level, we can combine the fullness of recycling and trash and display a total bin fullness. Furthermore, there will be more than one bin under a single location. By wildcarding under a specific location, we can collect data of how full each location is as a whole.

Other main topics like “ObjectDetect” and ‘RoutePlanning“ contribute to other key parts of DetritusAI, such as opening the bin lid and providing the drivers with step-by-step instructions of the optimized garbage collection route.

What We Learned

Communication between different components often takes more time than one might imagine. Thankfully, Solace PubSub+ is a very powerful tool that helped us dramatically reduce development time.

Through this project, we have learned:

• How to integrate several components together with Solace PubSub+ (React Frontend, Arduino MQTT communication, Python + MQTT)
• How to utilize Google Maps API to do route optimization
• How to use machine learning for image classification

What’s Next

In the future, we want to expand DetritusAI to have the ability to also classify organic waste as compostable. We also want to integrate a backend where we are able to keep historical data about the volume of garbage cans which we can use to further optimize our path planning algorithm.

Another improvement we would like to make is to the machine learning model used for waste classification. The model we used for this makeathon was very simple and was only capable of image classification. This caused occasional issues during our presentation as the model would get confused by the background from the camera feed. We would like to remedy this by using an object detection system such as YOLO which can distinguish objects from their surroundings.

Check out the video that demonstrates our project and explore our code on GitHub.

Authors