Robotics · Music Analysis · Interaction
Dabot is an interactive entertainment experience that brings a robot dog to life through dance. Using a commercially available Unitree Go1 robot dog paired with custom-built software, Dabot performs synchronised dance moves driven by two real-time inputs: a person's body movements and the music playing around it. The project demonstrates how algorithms can understand motion patterns, analyse music, and translate them into dynamic, natural-looking dance moves on a real robot — all in the name of fun.
A camera captures a person dancing in front of the robot. The video feed is processed on a PC using a pose estimation model powered by MediaPipe, which identifies the person's body keypoints in real time. This pose data is then passed through a custom algorithm that detects the patterns and frequency of oscillation in the person's movements.
Simultaneously, the system analyses the live music to extract features such as tempo, energy, and intensity. These audio features work alongside the motion data to inform the robot's behaviour, ensuring the dance feels connected to the music.
The robot has a predefined set of dance moves, but these are not played back rigidly. Instead, the current oscillation frequency from the person and the energy and intensity from the music are fed into an algorithm that decides: which move to perform, how intensely to perform it, and when to switch to a different move.
The result is a robot that doesn't simply mirror the person but performs synchronised, smooth motion that follows the dancer's rhythm and the music's feel. If no person is dancing, the robot falls back to an autonomous mode where it dances purely based on the music's features.
To enhance the visual spectacle, the lights on the robot are synchronised to match the music, adding another layer of immersion and fun to the experience.
| Component | Description |
|---|---|
| Pose Estimation | MediaPipe running on a PC, tracking body keypoints from a camera feed |
| Oscillation Detection | Custom algorithm converting pose data over time into oscillation properties |
| Music Analysis | Audio feature extraction for tempo, energy, and intensity |
| Control Interface | Desktop GUI for managing oscillations and robot behaviour |
| Communication | UDP over WiFi from the PC to the robot |
| Robot Control | ROS and the Unitree SDK running locally on the robot |
Dabot was showcased at Zuitzerland, a pop-up village in Switzerland that ran for a month. The team was invited to demonstrate their robotics project and connect with other creators and innovators.
The development of Dabot was largely a process of creative experimentation. While there were no major blockers, designing the algorithm and control system was far from straightforward. Getting the dance moves to feel natural required extensive trial and error — the team would tweak parameters, watch the robot perform, and judge whether it looked like a good dancer. It was a challenging but enjoyable process that blended engineering with artistic intuition.
The team plans to take Dabot to more events, bringing the experience of a dancing robot dog to new audiences and venues.