05 Feb In spite of Covid-19 situation, E-PILOTS keeps progressing
The Covid-19 situation is hitting really hard our lives, and E-PILOTS is not an exception. However, this project keeps rolling and, although all the difficulties and challenges it has been to face, the results after two years of its kick-off are very promising.
E-PILOTS is aimed to evaluate the potential of cognitive computing flight deck supporting tools, to preserve the situational awareness of the pilots while improving their performance in stressful flying scenarios. Scenarios where pilot cognitive tasks must deal not only with a procedure, but also interleaving with different interruptions such as the controller instructions, the Electronic Centralized Aircraft Monitor (ECAM) warnings and other emergent issues.
With these objectives in mind, the E-PILOTS consortium has designed a use case on a go-around decision support system, based on the probability of a hard landing event (when the acceleration at the moment of the touchdown is very high and it could damage the airplane), to validate the functional and non-functional requirements of flight deck intelligent supporting tools. For this purpose, two machine learning systems have been developed: a machine learning system to predict hard landings, and a machine learning system to continuously monitor in real-time the cognitive state of the pilot(s) and decide when is the best temporal window to provide elaborated information through extra perceptual variables in order to improve their situational awareness.
Regarding the first machine learning system, E-PILOTS has identified several variables related to physical magnitudes and actuators states, that can predict a hard landing during the approach phase. “This machine learning system is able to predict these events with a 78% of accuracy. This is a very promising result, taking into account that there are not so many systems able to do this. It is necessary, of course, to get higher accuracy, but we are already doing significant improvements”, Dr. Dèbora Gil, Computer Vision Center (CVC) researcher, explained.
For the second machine learning system, in order to monitor the cognitive state of the pilot(s), sensors such as electrocardiogram and electroencephalogram play a key role. “Taking measurements from electrocardiogram and electroencephalogram, we can differentiate when the pilot is in a baseline state –relaxed- from when they are doing or having a heavy workload and, depending on that, choose the right moment to communicate the situation to the pilot”, Dr. Aura Hernández, CVC researcher, described.
Dr. Hernández and Dr. Gil agreed that this machine learning systems is the crown jewel of E-PILOTS:
“The detection rate of the workload of the pilots is about 80%, a very high detection rate”, Dr. Dèbora Gil
Better than expected results
So far, the results are exceeding expectations. “The ability of detecting cognitive states from sensors could be helpful not only for pilots, but also for any specialized highly demanding jobs, like a clinician in the intervention room. Also it could be useful for developing decision supporting systems for people with special needs, like old people that starts to have mental cognitive problems”, argued Dr. Gil.
Dr. Hernández, from her part, wanted to highlight the unexpected results obtained:
“There are variables that we expected to be very influent in the hard landing, but our results suggest that they are not”, Dr. Aura Hernández
The Covid-19 impact
The main problem E-PILOTS has to face since its beginning is being the pandemic situation. “Until now, we could only conduct experiments in simulators with a reduced amount of volunteers, due to the difficulty to travel to UK where the consortium planned the use of Rolls-Royce Future Systems Simulator with the support of Cranfield University partner”, Dr. Hernández exposed. But this is not the only complication the Covid-19 is creating. “We could not deploy the systems in more realistic situations or test it with more pilots. Also, you need to place sensors in the chest to monitor the cardiac activity and a helmet with sensors in the head to get the signals related with the cognitive state, and during the first months of the pandemic this was considered very risky due to close interaction with volunteers”, delved into Dr. Dèbora Gil.
To try to solve these problems and push forward the project, E-PILOTS consortium is actively looking for ways to achieve more data in simulator experiments and be able to move forward till the final objective of this project, test the machine learning systems in a realistic Single Pilot Operation (SPO) flight simulation framework.
The pilots’ point of view
One of the main E-PILOTS’ strengths is the multidisciplinary nature of the consortium: experts in machine learning, aeronautical engineering, human factors, socio-technical simulation, research centres, universities, big private companies and, of course, professional pilots.
The main objective is, therefore, adapt all the systems to the pilots’ particular needs. For that reason, it is essential to know their point of view.
Jordi Manzano, pilot of Air Europa and teacher at the Universitat Autonóma de Barcelona, joined E-PILOTS because “it offers me the opportunity to learn and to obtain information about new tools and procedures that otherwise I would never know about”.
Mr. Manzano explained the reason why he decided to join E-PILOTS and his experience until now:
“E-PILOTS gives me the opportunity to be in direct contact with companies and experts that they have never been before a direct involvement with aviation”, Jordi Manzano
For Mr. Manzano, E-PILOTS contributes in many ways to real pilots, but he underlined that “the best part of being involved in E-PILOTS is to have the opportunity to learn and to propose different procedures for the aviation of the future”.
He has no doubt about that E-PILOTS tools will be useful in real fly situations in the future:
“I do not know how the aviation of the future will be, but I am sure that tools like E-PILOTS will be part of it”, Jordi Manzano
For his part, Miquel Gallego, pilot and flight instructor of the Barcelona Flight School, pointed out that “E-PILOTS gives me the chance to spend many hours in a flight simulator, facing on hard landing events, that for sure it is very positive for my training”
Also, he feels that his contribution is very useful for the development of the project:
“The point of view of the E-pilot is very helpful for the other participants in the project”, Miquel Gallego
Both of them are convinced that E-PILOTS could help pilots in real flight situations and expressed their willingness to use it beyond the simulator.