The dirt track leads past muddy fields and a farm on the outskirts of Lisbon to a hangar, which from the ageing signage and the old prop planes, was once an “Escola de Aviacao”. Inside the hangar a young Portuguese man sits at a desk and frowns intently at a laptop, concentrating on moving a dot either up or down the screen.
The man is wearing a maroon and blue cap, out of which trails bundles of wires from electrodes. The wires go to a computer running software that converts the electrical signals from his brain into instructions for an unmanned aerial vehicle (UAV) in the sky above the hangar. The dot on the screen goes up and the UAV goes right. Down and the UAV moves left.
About a dozen journalists and around the same number of staff from Portuguese aerospace company Tekever are witnessing the first public demonstration of a drone controlled by just the power of thought.
The demonstration is the result of more than a decade of research into Brain Machine Interfaces (BMI) at various institutions and companies in Germany, Holland and Portugal, and is the first proof-of-concept for the technology in a real world environment. Similar experiments have been done in the US, but with quad-copters, inside.
Ricardo Mendes, chief operating officer of Tekever, says: “Moving out from the simulated environment into a real environment is the main challenge. This is an actual plane, there is wind and rain. There is a lot of variables out there that are not present in a simulated environment. It's the first time this has been done, not just testing it for real, but showing it to the world in a live session.”
The Brainflight test site in Portugal
The two year EU-funded project developing this aeronautical application of BMI technology, called Brainflight, officially ended last June, with the successful demonstration of a flight simulator at Munich's Technical University. Tekever was the project's coordinator, and was so impressed with the progress made it has subsequently set out to prove that the BMI can control an unmanned aircraft.
The move from the laboratory into the real world is a big step. The weather during the test is bad by Portuguese standards - heavily overcast and very windy. As control switches to the thought-control pilot, the small drone, which is built by Tekever and usually used for security applications such as crowd control, is buffeted by 45 km/h winds. The dot on the screen moves, and half a second later so does the aircraft, but not for long. The pilot declares the conditions too windy. At one point the UAV is moving in the wrong direction. It's not the convincing display Tekever were hoping for.
It's also frustrating for the pilot, a post-graduate student on the project called Nuno Loureiro. He has trained for two weeks in a laboratory, roughly an hour a day, to learn how to control the UAV. He says: “The longer you train, the better the level of control. In the beginning, the protocol is like a game, you get tired before you get bored of the practice. The aim is to get like how you drive a car. You don't think about all the movements you make when changing gear, you just do it.”
The Electroencephalography (EEG) cap Loureiro is wearing detects slight changes in voltage between five different electrodes in contact with his head. This signal acquisition is the first of three main elements of the BMI. The off-the-shelf EEG cap, which is made by a company called Biomsemi, is normally used in the medical sector.
The EEG reading is sent to the second part of the system, the decoder, a computer running software written by the researchers. This converts the reading into an instruction and sends it to the UAV.
The final part of the BMI is the feedback, which through visual, audio or haptic cues allows the user to adjust his brain activity.
Rui Costa is principal investigator at the Champalimaud Foundation, a biomedical research organisation and partner on the project who has been involved with BMI research since its inception. He says the traditional approach to using EEG as a control system is to take a snapshot of a subject's EEG when performing an action, then to configure the interface to detect that action. For example the control of a robotic arm with the EEG reading of someone moving their arm. This has proved largely unsuccessful because different people produce different EEG readings for the same action.
Costa says: “Our approach is different. Our decoder is fixed. It uses the signal from the brain to create feedback and slowly the user learns how to control. In the beginning it may be all over the place, but slowly you learn. It's similar to the way babies learn to control their limbs through feedback.”
The feedback for the UAV is through a monitor and the ground control station. The pilot is given a series of missions, such as following a vehicle in a 3D environment, flying to a location or loitering in a certain area of the map, as ongoing training. Mendes, chief operating officer of Tekever, says the main technical challenge was to create all the signal filters and computer algorithms that transform the neural inputs into UAV command instructions. “We had to adjust and calibrate everything and create the appropriate user feedback inputs so the operator can learn how to control the UAV.”
The Tekever drone normally used in security applications
Benefits from the use of BMIs in aerospace could include a “dramatic” reduction in pilot training time, says Mendes. It could also free-up a pilot's cognitive functions to focus on the most important tasks. “The whole cockpit would have to be rethought,” he says. “It's designed to be manually controlled with buttons and levers. This changes the paradigm. But the technology can be applied to much more than aeronautics.”
However, the main goal of the project is to eventually enable physically disabled people to do tasks they couldn't do before, such as flying an aircraft. But regulatory hurdles mean it is unlikely to be used in manned aircraft anytime soon. Costa, from the Champalimaud Foundation, says: “For UAV's or for a cargo plane it's technically feasible. But there is strict legislation. You don't want anything to go wrong in the air. An airplane isn't like fixing a stuck wheelchair.
“The biomedical and everyday applications will come first, its possible within five years we will see the first things like wheelchairs. It's dramatic for people with paralysis. When we started working with people, just to turn on a computer and go through the applications, write an email, it's a game-changer for these people.”
However, one of the remaining major challenges is to improve signal acquisition. The better the quality of the signal from the brain, the better the command and feedback can be. “Noise” from outside sources is a major challenge for EEG devices, and the Biosemite cap is powered by a battery to combat this. The BMI's decoder is also resistant to noise, so you can chew or move your eyes and nothing happens. Costa says: “A system that could record and discriminate really well, with amazing signals like you get form intracortical recordings, would let us control multiple degrees of freedom simultaneously, or shift between tasks using different decoders.”
Improvements to brain signal acquisition devices that make them easier to wear and use would also help break into mainstream markets, Costa adds. They could be used to operate household devices such as TV's and lighting. They are especially useful in environments where you have to be silent. Another possible application is security. The Champalimaud Foundation has been approached by a security firm to help develop a system that will enable camera surveillance operators, who sometimes have to view hundreds of feeds simultaneously, to switch cameras using brain activity.
The demonstration over, it's difficult to deny the viability of the technology, despite the windy conditions. However, the man-hours of engineering work required to get BMI's off the ground remains considerable.