Pedro Cruz spent weeks after Hurricane Maria ravaged Puerto Rico in September 2017 helping bring food and water to people trapped in remote areas.
He quickly realized he could use an airborne drone to help, using its video connection to read dozens of messages painted on the ground asking rescue crews to bring water, food or medicine.
It wasn’t until nearly a year after the hurricane devastated the island territory in September 2017 that Cruz figured out a way to connect his drone to disaster aid through a computerized visual recognition tool.
Almost by luck, he said in an interview with FierceElectronics, he learned about an IBM Call for Code hackathon being held in Bayamon, Puerto Rico, in August 2018. Developers were asked to find tech solutions for natural disaster preparedness, and as a self-taught developer, Cruz decided to join up.
Cruz ended up winning first place at the hackathon for a tool he introduced and later developed into DroneAid.
It uses visual recognition to detect and count emergency icons like SOS on the ground from video streams overhead. Then, it automatically plots the emergency needs on a map for first responders.
Following the hackathon, Cruz further developed DroneAid and later became a full-time developer advocate for IBM. On Wednesday, IBM also made DroneAid an open source project as part of its Code and Response initiative, a $25 million program to encourage development of open source technology designed to address global problems like disaster relief.
“Our team decided to open source DroneAid because I feel it’s important to make this technology available to as many people as possible,” Cruz said in a blog posted Wednesday.To help the project grow, people are being asked to contribute on a GitHub site.
In the interview, Cruz described DroneAid as a “passion project” that was informed by six months of living without electricity following the Category V hurricane. To this day, two years later, the stoplight outside his apartment in San Juan still doesn’t work, and there are frequent blackouts, especially in rural areas. Many houses in poor communities are still covered with blue tarps.
As a freelance web developer, he couldn’t reach clients for weeks after Maria hit. Just afterwards, he used his drone to locate his grandmother who waved from outside her isolated home that she was doing OK. Two weeks after the storm passed, “we would go out to the mountains in the center of island and it still looked like the hurricane had passed just two days earlier…That’s where the inspiration for DroneAid came from. With a tool like this we can make our response a lot faster and many organizations can go out and help.”
Weeks after the hurricane passed, Cruz’s grandmother was hospitalized with a respiratory condition and later died. He later dedicated DroneAid to her memory.
Cruz plans to work from the bottom-up to get more people trained on using drones for emergency response. He has also worked top-down and has reached out to San Juan officials and the Red Cross. He hopes to talk to leaders in other cities about drone responses for all kinds of natural disasters.
“My vision is for Puerto Rico is to have an automated fleet of drones,” he said. “I see in the near future a fleet in certain areas, used right after a disaster to report to first responders. That would cost money, but that’s where the benefit would be.”
There even could be a system for people to share their needs for food or water via Twitter or other social media—without any drone—and be located via meta data and GPS.
One discovery Cruz made early on was that artificial intelligence computer vision systems needed to read a standard set of icons asking for assistance instead of reading handwritten messages on the ground in various languages through optical character recognition. He settled on eight different icons—such as SOS, OK, food, water, medicine—drawn from a recognized set of icons used by the United Nations. They can be printed on mats that are distributed prior to a storm or spray-painted or drawn by hand.
Cruz explained that a drone can survey an area for the icons placed on the ground by people in need or community groups. As DroneAid detects and counts the images, they are plotted on a map in a Web dashboard to help first responders prioritize needs. The AI model has to be trained on the standard icons to be able to detect them in low light and faded conditions. For this purpose, IBM has a cloud annotations tool (found on the GitHub site) that relies on IBM’s Cloud Object Storage.
When the AI model is applied to the live stream of images coming from the drone, each video frame is analyzed and if any emergency icons are found, their location is captured and plotted on a map. Any drone that can capture a video stream can be used.