BCI · VR · Robotics
Reach is a brain-computer interface (BCI) system that lets users control a quadruped robot entirely hands-free — using only their brain signals. The user sees the world through the robot's eyes in VR, selects actions by focusing on visual markers, and commands the robot in real-time across a network. No joystick. No controller. Just thought.
The name says it all — Reach is about extending a person's physical capability beyond their body. Built with assistive use at its core, the system is designed to help people with disabilities navigate spaces, identify objects, and eventually retrieve them using a robotic arm.
Controlling robots with traditional interfaces — gamepads, keyboards, joysticks — demands manual dexterity that not everyone has. Existing BCI-controlled robot systems tend to be lab-bound, high-latency, and disconnected from the user's sense of presence. There's a gap between what BCI research demonstrates in controlled settings and what would actually feel usable to a real person.
Reach addresses this by combining three things: a practical BCI paradigm (CVEP via NextMind), an immersive first-person VR view, and a robust quadruped robot — all connected through a low-latency streaming pipeline.
Reach grew out of Ronald Lanton's thesis research, which used a 64-channel EEG system to decode brain activity associated with imagining the forward and backward motion of a robot. The thesis explored whether motor imagery could serve as an intuitive control signal for robotic navigation.
That work revealed both the promise and the friction — EEG-based motor imagery decoding was possible, but far from practical for real-time control. Reach was born as the next step: a system that moves closer to a feasible, deployable control modality using a commercially available BCI, immersive VR telepresence, and a robust robotic platform.
The system forms a closed loop between the user's brain and the robot's body:
See — A camera mounted on the Unitree Go1 streams live video to the user's VR headset, creating a first-person view (FPV) similar to piloting an FPV drone. The user sees what the robot sees.
Decide — Inside the VR environment, NextMind CVEP markers are overlaid on the scene, each mapped to a specific robot action. The user simply looks at the marker corresponding to their desired command.
Decode — The NextMind BCI reads the user's brain signals and decodes which marker they are focusing on using code-modulated visually evoked potentials (CVEP).
Act — The decoded command is sent over UDP across the network to a ROS node running on the robot, which translates it into velocity and rotation commands via the Unitree SDK.
Using the NextMind markers, the user can perform six core actions: walk forward, walk backward, rotate clockwise, rotate anticlockwise, sit, and stand. The command format sent to the robot is a simple string: [x-axis velocity], [rotation].
Beyond navigation, Reach also features intelligent object interaction. YOLO v11 runs computer vision to detect objects in the robot's camera feed. Detected objects are overlaid with NextMind markers in the VR scene, allowing the user to select a target object simply by looking at it. The robot can then autonomously navigate toward the selected object.
One of the core technical challenges was achieving low-latency video streaming from the robot to the VR headset. The solution uses NVIDIA hardware-accelerated H.264/HEVC compression via GStreamer, streamed over the network and rendered in Unity using the MRay Unity plugin.
The Meta Quest 3 connects to the PC running Unity via a Link Cable, keeping the visual pipeline tight and responsive. This architecture balances visual quality with the real-time responsiveness essential for effective telepresence.
To measure how effectively users can control the robot, a structured evaluation course was designed. Numbered flags are placed sequentially along a path, and participants must navigate the robot from start to finish using only the BCI-VR setup.
A drone tracks the robot's trajectory from above using marker tracking. This actual trajectory is then compared against the intended course path to calculate deviation. The study aims to collect data from multiple participants to understand, on average, how intuitive and accurate this control modality is across different users.
Reach has been demonstrated at multiple international builder communities: Zuitzerland — a curated popup village bringing together selected builders and technologists, and Zugrama — a popup village gathering of innovators.
