RoboEtude
Compare robot pianists on 25 excerpts, from five-finger drills to Liszt. Hear each performance alongside its target.
Controls and training
Note-by-note comparison
- Target
- On time
- Early / late
- Extra
- Missed
On time: within 50 ms. Early / late: within 150 ms. Select a point on the roll to seek.
Leaderboard
25-piece scores
How to read this
Overall = 60% notes + 20% release + 20% loudness. Each level counts equally. Bars show 95% confidence intervals; overlapping intervals do not establish a clear ranking. A level passes at 80, in order.
Notes measures pitch and onset; release measures sounding duration, including pedal; loudness measures strike velocity. The last column describes the main measured weakness. Select a system to watch.
| # | System | Overall 95% CI | Notes | Release | Loudness | Passed | Main limitation |
|---|
Hear CANTABILE on its training edition of Chopin’s Nocturne
Scores by level and system details
| System / setup | L1 | L2 | L3 | L4 | L5 | Unseen | Frame F1 |
|---|
Unseen: sealed-set score for the evaluated controller version. Frame F1 is reported separately and does not affect the score.
Test a controller
Download benchmark kit (.zip)Play any of the 25 excerpts on the shared hands, or submit measured key and pedal events from your own simulator. The kit includes the scorer, a working controller and a baseline you can replay on every piece.
How we test general-purpose models
In the controller-writing test, the model hands in its first program before trying the simulator. It can then practice for a fixed time and revise the program. Each version plays three exams. The direct-control test gives the model a new observation every quarter second and asks for five joint commands.
Both paths have been tested with scripts. Controller-writing checks · Direct-action checks
Submit a performance
Run the kit’s replay check. Share a downloadable folder or repository containing the controller or native commands, scene instructions and measured events. Reviewed replays can appear on the leaderboard.
Method and sources
Every system plays the same timed excerpt on an 88-key piano. Scores come from physical key strikes, releases and sustain. The setup beside each performance identifies the hands and the practice allowed.
| Component | Measure | Weight |
|---|---|---|
| Notes | Correct pitch and onset | 60% |
| Release | Correct sounding duration, including sustain | 20% |
| Loudness | Correct strike velocity | 20% |
Scoring and evaluation protocol
Scoring
Each component is F1, averaged over onset tolerances of 50, 100 and 150 ms. Misses and extra strikes reduce the score.
Release tolerance is 20% of the target’s sounding duration, with a 50 ms minimum. Loudness tolerance is 8 MIDI velocity steps. Both require a matching pitch and onset. Sustain affects target and played durations equally.
Runs are averaged per piece, pieces per level, then the five levels equally. Levels pass in order at 80. Confidence intervals resample pieces within levels.
Evaluation
Three runs per piece: one clean, two with seeded command noise. The stage uses MuJoCo, RoboPianist and CANTABILE’s key-velocity model; 38 joint targets plus pedal at 20 Hz, with hand collisions enabled.
PianoMime uses its own hand model with inter-hand collisions disabled, no simulated count-in, and one scored run per piece. Its released policy has no loudness or pedal control. One failed exercise scores zero.
Twinkle and Chopin’s Nocturne use the RoboPianist authors’ recorded hand movements. We trained a separate policy for each Level 1 and 2 drill and for the opening of Liebesträume. The original Twinkle recipe and the full-update specialist have separate entries.
The public setlist allows practice. The sealed set contains 100 unseen pieces and reports aggregates only. Real hardware and listener preference are not evaluated.