Intel’s Mobileye has been testing two autonomous vehicles in New York City for about six weeks, taking on the challenges of jaywalkers, crowded intersections, driving through tunnels and construction zones, double parking and aggressive drivers.
“Using New York for battle testing is very, very challenging,” said Mobileye CEO Ammon Shashua in comments to reporters on July 19. “It’s a huge headache to test in New York City.”
The vehicles being tested rely on 12 cameras and two EyeQ SoCs along with a data crowdsourcing program called Road Experience Management (REM) which uses real-time data previously gathered from nearly 1 million Mobileye-equipped vehicles to build out a global 3D map.
Mobileye also has a separate system under evaluation that relies on lidar and radar and no cameras that will be combined with the camera system later this year.
Shashua offered a few examples of the challenges of New York driving, including detection of red lights at crowded intersections with pedestrians crossing from various directions. Detecting whether a signal is red, yellow or green is not hard, he said.
What is more difficult is for the AV to know which light is relevant, where there might be five lights facing the car with one for right-turns and another for left turns. “That’s very difficult to understand in real time in a very busy environment,” he said. “We’ve been working on traffic light detection for the past five years. It’s tricky but now in a very advanced state.” The Mobileye test vehicles have encountered many traffic lights “but manages them quite well.”
When the AV approaches an intersection it already knows the location of lanes, curbs, crosswalks, signals and signs from the REM system data for each specific intersection. “Except for the road users, you know everything at a high level of detail,” Shashua said. “What is left to understand is where are the road users. The color of the traffic light is the least of the problems.” The data provides drivable paths in a city and the association between them down to a centimeter.
In coming weeks, seven more vehicles will be added to the New York Mobileye test. Intel released a 40- minute unedited video of one AV drive through New York to show the variety of challenges the AV system can face. A safety driver is seated behind the wheel but doesn’t touch it. (Mobileye had to get a special permit from New York to permit its safety driver to be completely hands free.) The video shows what the computer vision depicts, with boxes around obstructions and vehicles and a red line along areas the vehicle cannot cross. There is also a camera on the roof recording what the car sees from a human perspective.
Shashua said that Mobileye is the only company pursuing both development or robo-taxis and consumer AVs. The company also has a goal of driving costs for various components in each vehicle below $5,000. Tesla is clearly headed toward developing a consumer AV and already has similar computer vision approach, he noted. Apple, if it comes up with a car, will probably develop one for consumers, he said. In early 2021, Mobileye committed to launching a fully driverless delivery vehicle service in 2023.
The 12 camera system being used in New York vehicles will go into production as a Level 2 advanced driver assist system on Zeekr, an electric vehicle from China-based Geely.
China and much of Europe as well as Israel are moving faster than the U.S. on autonomous vehicle technology, Shashua noted. “The U.S. is more complicated,” he said, partly because regulations vary across the state and federal level. “Once we see progress outside the U.S., that will help with convergence of [regulations]. I believe it will happen and I’m optimistic,” he added.
Sashua said New York driving is among the most difficult in the world, worse than Boston.
“If you can conquer New York City and Tokyo and Tel Aviv, it opens up a big part of the world,” he said. “Driving in India, I don’t see us doing that in the next decade.”
RELATED: Mobileye expects consumer AV in 2025 with cameras, radars and lidar
RELATED: Intel’s Gelsinger: M&A not critical, ‘nor would we rule it out’