In addition to cameras, GPS, geofencing and other sensing techniques to keep its autonomous tractors safe and on track, John Deere has also resorted to hiring actors to stand in fields and be photographed so its AI algorithm will know to tell a tractor under power to avoid .people.
Deere has hired actors of different shapes, sizes, and races to stand in various crop fields with shadows and bright sun, in front of trees and fences and in various poses. “We classify humans to train the machine not to operate around humans,” said Joseph Liefer, senior product manager for autonomy at John Deere.
“The number one goal of our autonomy research is safety and the number two goal is productivity,” Liefer added. “Customers expect we will have a safe product.”
While the notion of using actors to help train AI might sound whimsical, it is one more step in a laborious process of training the Deere AI autonomous driving dataset that has grown from 50,000 images to 250 million images over the past five years, largely with the addition of images from cameras on customers’ tractors around the globe.
Deere has taken many images of its own to train its AI, including photographs of thousands of varieties of weeds in fields to use with its See and Spray product for precise targeting of weeds with pesticides.
AI can act by shutting down the tractor if it encounters something it does not recognize, but Liefer said Deere decided it must be more definitive by training the AI on something to avoid, namely humans.
Deere also loads in maps of areas to be plowed or harvested so the AI knows where the creeks and waterways are located. An autonomous tractor will also be assigned a geofence at the edges of a field before it ever takes off, and then the AI will calculate the best, most efficient way to work the field which also would be the safest.
In addition to all the AI training and inference in its system, safety is enhanced by the speed of its equipment, which move no faster than 14 mph. They are not authorized to be operated on roadways and must be driven by a human to a field before a farmer starts the tractor remotely.
Enhanced lighting all around and six pairs of camera sensors are used to help the tractors function in the dark. Lidar and radar are possible sensors for future applications and might come into play as Deere produces smaller tractors for farms.
Deere officials know that farmers might be slow to adopt the autonomous concept, but the company already has paying customers signed up for buying new tractors in the fall and expects many more will be retrofitted in coming years. “Autonomy is just getting started,” said Deanna Kovar, vice president of production and precision ag production systems. Autonomous Deere machines have been in trials on farms in the upper Midwest for much of 2022.
The company has already had technology in place for years to allow its tractors to precisely move row by row through a field automatically, with a farmer seated behind the wheel. By contrast, autonomous capability means the driver does not have to be sitting in the cab and can be monitoring it while working or relaxing elsewhere from a connected smartphone, tablet or computer. This frees up the farmer to take breaks, and finish work in busy harvest or planting times, which is especially valuable amid an ongoing shortage of labor, Liefer said.
The autonomous tractor is not necessarily more productive than the current automated tractor for each hour worked, but the productivity advantage comes when it can be used for more hours than it has been traditionally when the farmer or a farm hand is not available, Liefer said.
RELATED: John Deere wants to connect its tractors and other machines to satellites