The Autonomous Oxymoron: HMI Development for a Self-driving Vehicle

Lamec Fletez, Lead Engineer-HMI and UX Development, LM Industries

Lamec Fletez, Lead Engineer-HMI and UX Development, LM Industries

Before we begin, let’s address the elephant in the title. Why worry about HMI for a vehicle that doesn’t need a human to operate? However, a human and machine interface is a necessary ingredient for an autonomous people-mover. We aren’t talking about your grandad’s HMI; the industry needs to change its outlook. We aren’t building a traditional HMI for the safety of the pilot but for the passenger and the world around it. An autonomous vehicle should be able to assure people that it’s safe, keep the user entertained, productive, and communicate with the world around it. It even needs to stand up for itself. Its passengers, surrounding motorists, and pedestrians outside of the vehicle will depend on it.

LM Industries brought me on last year to lead the HMI and user experience development of the next generation Olli, the world’s first cognitive electric autonomous vehicle. With my background in software development, I found that the same methodologies used to lead software projects were useful in the development of Olli’s HMI system. It has been important to treat Olli’s new HMI as a separate product, much like a new digital experience. The slate of autonomous vehicle development needs to be wiped clean to make way for designers and developers who are ready to provide solutions similar to consumer grade digital experiences instead of typical auto industry solutions.

Why is the investment in a strong HMI so important? Perception and comfort

We constantly hear about how traffic will decrease once autonomous vehicles take over; however, that won’t be for some time. We also like to talk about how much safer transportation will be once autonomous vehicles are more common. While this is true, traffic deaths will still exist until we completely eliminate the human factor, which unfortunately won’t be for some time. What we, as a community of UX experts and HMI devs, need to do is mold our passengers’ minds to perceive that self-driving vehicles are safer than traditional vehicles. Not only that but also the idea that autonomous vehicles can be a time for more productivity. How do we convince the populous that we are safer in autonomous vehicles? How do we help them believe we are arriving at our destinations having accomplished things that we normally would not have been able to before?

"There are endless possibilities for productivity and entertainment applications when a self-driving car becomes another connected device that your smartphone or wearable device can interact with"

Reinforcing the safety of autonomy by molding the right perception

We can enable trust and a feeling of safety for passengers new to this experience (ahem, we’re all new to this from an experiential standpoint) by visualizing the autonomy suite’s sensor data. When we drive, we scan our viewable area for potential hazards; the autonomy kit in self-driving vehicles can do the same thing. We can take the data being “viewed” by the sensors and display it for the passengers to view. By simply classifying the objects by size or distance, passengers will recognize the vehicle’s constant predictive observance and avoidance. This will enlist trust in the system and enable the passenger to enjoy their ride.

Drive-time productivity and entertainment

Self-driving vehicles will still be subject to humans regulating traffic. We should expect travel times to remain the same for at least a little while longer. We can still increase the value proposition regarding time even if it isn’t shortened. This is a classic elevator redesign problem. The problem doesn’t have to be solved by increasing the speed to arrival but instead by what we allow our passengers to do while traveling. A self-driving car is nothing more than another IoT device. It’s full of sensors, it knows its location, and we should know who is embarking. There are endless possibilities for productivity and entertainment applications when a self-driving car becomes another connected device that your smartphone or wearable device can interact with.

Handling outside variables: Pedestrians

Unfortunately, humans can be bullies. We have to anticipate the bullying of robots and other autonomous systems, including self-driving vehicles. Vehicles like Olli that don’t have a primary operator like a Waymo minivan could be susceptible to bullying. There is no bus driver to shame a pedestrian taking their time crossing the street or to stop someone from jumping in front of a vehicle just to test the limits of self-driving technology. Just the acknowledgement of pedestrian’s presence is enough to force (most) people to fall in line. External HMI systems could accomplish the same thing. Who knows, maybe placing a mirror in the direction of crosswalks could ward off these behaviors. I haven’t tried this experiment yet. Send me your findings if you try this.

Handling outside variables: Other motorists

Another important element that’s missing when not having a human operator is driver-to-driver acknowledgement. In the future, self-driving vehicles could work as a hive-mind to keep traffic flowing efficiently. Currently, however, drivers have to use each other’s body language to promote traffic flow. If a person meets an autonomous vehicle at a four-way stop at what is perceived to be the same time, who turns or proceeds first? How would the driver of the ordinary vehicle know that if they proceed the self-driving car won’t drive into them? Exterior HMI systems can help communicate intentions.

Although futuristic 80’s movies anticipated society to be hovering around in self-driving cars at this point, we have a long way to go in the development of our autonomous vehicles. If you’re venturing down this same labyrinth I am, I hope you have as amazing a team of engineers and vehicle designers as I do.