BCI · VR · Drone
Mindrone is an experimental project exploring how Brain-Computer Interfaces (BCI) and Virtual Reality can make drone control more intuitive. Instead of thumbsticks or touchscreens, the pilot wears an EEG headset and a VR headset — look at a command to fly, turn your head to steer. It's drone piloting that feels less like operating a remote and more like being the drone.
This project is part of a broader effort to explore natural control modalities for external systems. The long-term vision? Mount the drone on a Unitree Go1 quadruped robot, fly it from the robot's back to scout obstacles from above, map the area in VR, and then seamlessly switch control back to the ground unit.
The pilot wears a Nextmind EEG device that reads brain signals using code-modulated Visually Evoked Potentials (cVEP). In the VR scene, visual markers are displayed for core commands — takeoff, land, forward, and backward. The user simply focuses their gaze on a marker to trigger the corresponding action.
Heading control is mapped to the pilot's head rotation via the Meta Quest 3. Turn your head left, the drone yaws left. It's a direct, natural mapping that clicks instantly.
The live camera feed from the DJI Tello drone streams in real-time into the Unity VR scene using GStreamer and the MRay plugin, so the pilot sees exactly what the drone sees.
The system runs on a PC with the Meta Quest 3 tethered via Quest Link cable. Unity handles the VR environment, Nextmind SDK integration, and video stream rendering. When a BCI command is triggered or the head orientation changes, Unity sends a UDP message to a Python program, which relays the command to the Tello drone over Wi-Fi using the DJI Tello Python SDK.
The biggest design challenge was cVEP marker placement. The markers need to be visible enough to generate a reliable brain response, but they can't block the drone's live camera feed. Getting that balance right took several iterations of testing and repositioning until the team landed on a layout that was both functional and usable.
The next step is integrating Mindrone with the existing Unitree Go1 quadruped control system — including building a launch and landing pad on the robot's back. The goal is a single operator switching between ground and air perspectives using BCI and VR, all in one seamless workflow.