Jonathan Newell finds out how Nissan is taking the realms of vehicle connectivity to new heights by plugging the driver’s brain into the equation.
A fundamental requirement for autonomous vehicles of any kind is connectivity and modern cars are becoming increasingly connected both internally between sensors and control units as well as externally so that the vehicle can make sense of its surroundings and can make decisions based on such things as signals, weather conditions and the presence and position of other vehicles.
The industry coined the generic “V2X” term to mean “Vehicle to anything” communications, Whilst initially, the “X” represented other vehicles or infrastructure, the abbreviation has since spawned other variants as connectivity increases, such as Vehicle-to-Grid (V2G) to describe the relationship between electric cars and the electrical distribution network.
At this year’s glitzy Consumer Electronics Show in Las Vegas, Nissan surprised many and shocked a few with its announcement of Brain-to-Vehicle (B2V) connectivity linking the driver’s thought patterns to the vehicle control unit.
Whilst the ultimate aim of autonomy is to disconnect the human entirely from the process of driving the car, Nissan is sure that gaining an understanding of human thought patterns whilst behind the wheel will aid developers to create more reliable and human-like autonomy.
Meanwhile, even without fully driverless control, linking the brain to vehicular logic will help the car to learn human responses and provide predictive control functions, such as priming the brakes in anticipation of the driver’s foot reaching the pedal.
Brain signal interpretation
Nissan describes its B2V technology as a method of enabling the vehicle to interpret signals from the driver’s brain and thereby redefine how people interact with their cars.
The technology is expected to speed up reaction times for drivers and will lead to cars that keep adapting to make driving more enjoyable. The key to the usefulness of the research is the adaptability of the vehicle control unit resulting from the interpretation of driver brain signals.
All drivers react differently, particularly in difficult conditions. On the other hand, computer control units are designed to react predictably and repeatably, something which isn’t the best approach in a task that has so many variables and which relies to a large extent on judgement rather than pre-defined algorithms. Understanding driver reactions in the most difficult situations can help machines to learn behaviours in a more adaptable way.
Drivers don’t always make the correct decisions and with a wealth of measured responses, there’s the opportunity for machines to learn from mistakes and build-in “safe option” judgements for those difficult decisions that sometimes make drivers indecisive.
The B2V technology from Nissan is the result of research into using brain decoding technology to predict a driver’s actions and detect discomfort, essential aspects in learning behaviour patterns.
Predict and detect
The technology works on the basis of two principles – Prediction and Detection.
The system can predict actions by catching signs that the driver’s brain is about to initiate a movement, such as turning the steering wheel or pushing the accelerator pedal. Once this prediction has been made, the control unit can instruct driver assist technologies to initiate the action more quickly than the driver can. This can improve reaction times and enhance manual driving.
As well as predicting driver responses, the B2V technology is also able to detect and evaluate driver discomfort. Whereas the predictive principle is linked to providing automated assistance when the driver is in control, the detection principle is associated with autonomous control, learning about driver discomfort and deploying artificial intelligence to change the driving configuration or driving style when in autonomous mode in line with driver expectations. In this way, for example if the driver feels the speed is too high for the road conditions, the autonomous control system will detect this and slow down.
Improving the ambiance
According to Nissan, the use of B2V technology isn’t just a matter of improving autonomy or delivering a safer driving experience, but also about improving the quality of time spent in the vehicle.
Dr Lucian Gheorghe, senior innovation researcher at the Nissan Research Center in Japan, is leading the B2V research efforts and says that the technology can be used for making adjustments to the internal environment of the vehicle.
He sees the current research as being a catalyst for more innovation inside the company’s vehicles for the foreseeable future.
“The potential applications of the technology are incredible. For example, the technology can use augmented reality to adjust what the driver sees and create a more relaxing environment,” he explains.
Daniele Schillaci, an Executive Vice President at Nissan, sees the technology as being the antithesis of impersonal autonomy, being instead something that immerses the human more deeply in the driving experience.
“When most people think about autonomous driving, they have a very impersonal vision of the future, where humans relinquish control to the machines. Yet B2V technology does the opposite, by using signals from their own brain to make the drive even more exciting and enjoyable,” Schillaci says.
Nissan Intelligent Mobility
B2V is part of the wider Nissan Intelligent Mobility programme, a set of research and development projects on autonomy, electrification and connectivity that has already established Nissan as being one of the main players in the new wave of Connected Autonomous Vehicle (CAV) technology.
Nissan is pursuing three routes towards changing the concept of vehicles as we know them today. Intelligent driving is the route the company is taking towards autonomy, intelligent power is focussed on electrification and intelligent integration aims to establish new roles for vehicles as an integral part of society that contributes to the improvement of people’s lives, beyond just being a mode of transport.
From Mars to Urban Landscapes
Nissan has been working in partnership with the NASA Ames Research Center to explore the technology transfer potential of the space giant’s experience of autonomy on Mars Rover projects.
The car manufacturer is using the knowledge gained from NASA to develop collaborative networks of autonomous fleets in difficult city environments.
Maarten Sierhuis, director of the Nissan Research Center in Silicon Valley says, “Our goal is to deploy the technology to help third parties safely integrate a fleet of autonomous vehicles in unpredictable urban environments, for example ride sharing services, public transport or logistics and delivery services. The final stage of our existing research agreement with NASA will bring us closer to that goal.”
NASA sees the project as an opportunity to take space research and deploy it in ways to bring benefits to society. “One of NASA’s strategic goals is to transfer the technology developed to advance NASA mission and programme objectives to broader commercial and social applications,” says Eugene Tu, NASA Ames Director. “Using NASA’s work in robotics to accelerate the deployment of autonomous mobility services is a perfect example of how the considerable work required to advance space exploration can also pioneer advances here on Earth.”