Peter Hartwell holds more than 40 patents and has been CTO for seven years at Invensense, a TDK Group company, based in San Jose, California. His operation oversees 600 engineers who have developed a broad range of technologies and sensors for drones, automotive, industrial and, more broadly, IoT. He spoke to Fierce Electronics after Sensors Converge 2022, touching on AR, VR, robotics, drones, sensor fusion, the engineering workplace and more. The following is lightly edited.
FE: How was Sensors Converge for you and TDK?
PH: It was nice to see everyone back after Covid. I’ve been to Sensors several times. For some reason, I saw a lot of wireless power transfer tech in the booths on the show floor. At TDK we do that well. People obviously are interested in wireless power, thinking they’d buy more sensors if they didn’t have to plug them in. However, I’m still not sure what wireless power is going to do for us. With IoT, we want to have devices and not worry about the battery for years. Energy harvesting is a tough problem.
FE: What are you seeing as big sensor trends?
PH: With AR, VR and robots, for me, it’s the same problem with two sides of the coin. With robots you are interacting and moving around in the visible world. And with AR and VR you are taking humans into the digital world. So, robots need to see the world including obstacles: how am I moving, where are my extremities. We build them mostly like humans with stereo vision. With VR, I need sensors to record the world and play it back and record more inputs and motion and use that to manipulate the visual world and fool the senses with a display and sound for ears and with haptics for people in chairs. Sensors with digital transformation are trying to make that boundary disappear. The device is anticipating what you are trying to do, where you are looking That device is ready to go when you want to do something.
So VR has a strong future. With metaverse, we are most interested in the content creation side, not story content like Disney or Warner Brothers but what can be done with tools and sensors. Metaverse is not about playing Fruit Ninja. It’s about being able to preview a hotel room and do I want to upgrade the room or before going to a restaurant do I want to reserve the table by the window. You are going to be able to change travel and shopping and sports. I have been able to use a 360 degree camera with gyros and sensors while skiing and showed by father at 80 what it is like to ski with his grandson. So it’s not about gaming, but more about collecting and sharing experiences. If that’s cool, then I will pay money to do that for me. In my case, I will know that sky diving is not for me.
VR is now like the early days of TV when vaudeville was used for TV content. It took TV 70 years to get to Game of Thrones. With tools it will be similar to how Go Pro launched YouTube, so sensors and other tools will create VR experiences. My guess is that we’ll eventually see a platform where Disney or Warner Brothers offer full two hour experiences. Until that time, when I show my VR content, it will be for somebody I know like, “Let’s go shopping on Rodeo Drive with what’s-her-name and she will show you what you want to look at.”
FE: Tell me more about TDK’s part in all this.
PH: We have tools ready today to create content where sensors need to be and to record that real experience. Today’s 360-degree camera offers a linear video through time and you are flash stitching that video together...but it can be too hacky and has to go. The things we are working toward are hardware and software for how to enable the world of robotics and AI and the VR world. The tools are going to be loaded with sensors. For a robot to be safe and useful it will be loaded with my sensors. If I am creating VR, I don’t want to worry about the gyro in the camera or if I have enough resolution. We now dominate in the AR/VR and drone market and we are good at controlling with image stabilization and gyros.
FE: Now that you mention it, what’s happening with drone tech and the drone market?
PH: The drone market is morphing into the autonomous robot market. It turns out that flying is easier because there are mostly no obstacles to avoid in the air. We’re seeing drones as a platform for autonomous pilots, with modules for autonomous ground vehicles and vehicles in the water as well. We’re seeing an explosion for out of line of sight and other autonomous vehicles.
FE: But what’s the status of drone flights beyond line of sight and regulatory acceptance?
PH: The industry is moving closer, but it will take millions of hours of safe flights to prove drones can operate safety beyond line of sight. Technologically, I worry about traffic control and that’s the ultimate question: How many drones can safely fly in the same air space?
FE: We’ll need software management tools, like terrestrial road and traffic tools?
PH: Yes, I love these tools. When we see fog roll in in San Francisco today and 60 passenger planes drop to 30 an hour, I really can’t imagine 100 drones up there. So, it’s going to take time and it will be driven by necessity. We’ll need to see how many Amazon packages you can put on a drone. The answer isn’t coming any time soon.
FE: What’s the state of sensor fusion?
PH: A big piece of our portfolio is microphones and motion sensors converged in stereo wireless headsets. With spatial audio, you can track the head. When I’m talking I want the sensor to pick that up from the vibration of the jaw to lower the music. So it’s a strong fusion of motion and audio and for us it’s just great to put two businesses into one and really go after that market. It’s growing so fast. We understand low power mics that are always on. It’s exciting. We have digital mics and on-board acoustics and voice activation detection. We have the Six Axis ED&P, an edge digital motion processor with a custom core using extremely low power. That first launched all the way back to 2010.
FE: Big picture, how are you dealing with supply chain worries of the past 18 months?
PH: We’ve weathered the storm well. We can’t keep up with demand and have grown immensely through it all. We’re fabless and have managed well, so it’s been good for us. We outsource assembly and get wafers in Taiwan and test in house.
FE: And how are you dealing with the common problem of attracting and keeping engineering talent?
PH: We’ve adjusted really well to the new way of working. We’ve been a global company with design centers in Europe and Taiwan with 600 engineers so we’re working several time zones and it’s smoother and virtual. We’ve found good talent by leveraging our regions well. Silicon Valley engineers are moving around, but globally it’s a different story. We’ve found that what we do is unique and I can always find people to work on that part of the future. We have our products in Nintendo and PlayStation and so it’s easy to get inspired here. Engineers like challenges and sensors are tough.