RysUpAlign 2.1 puts real choir doubles 14.9 ms from the guide at the median. Doing nothing leaves them 34.8 ms off. That is the headline from a vocal alignment benchmark we built, ran, and published in the open, because nobody else had one.
Plenty of plugins say they align vocals. Almost none show you a number. So we made the test, ran the best public research models through it, and put RysUpAlign 2.1 next to them. Here is every result, the good and the honest.
Disclosure: Rys Up Audio makes RysUpAlign. We also built the benchmark. That is exactly why the data, the scoring code and the test cases are public, so anyone can check our work.
The results at a glance
Short version: 2.1 is a different engine. The old 2.0.5 engine was roughly no better than doing nothing on real choir singers. 2.1 beats doing nothing by a wide margin and beats every public model we tested on our main real singer suite.
What timing error in milliseconds means
Every note in a double starts and changes pitch at some moment. The guide vocal does the same thing at its own moment. Timing error is the gap between those two moments after alignment, measured in milliseconds (ms). One second is 1,000 ms.
Lower is better. Zero means the double hits every note change exactly with the guide.
Why it matters: gaps of a few tens of milliseconds are what you hear as flams and smeared consonants in a stack. In our real choir set, singers sat a median of 34.8 ms apart before alignment. Pulling the typical note under 15 ms is what makes a stack sound like one big voice instead of four people guessing.
We report two numbers. The mean is the average error, so a few badly missed notes pull it up. The median is the middle note, which tells you what a typical note sounds like.
RysUpAlign 2.1 benchmark results
| Test | RysUpAlign 2.1 | Best public model tested | Doing nothing |
|---|---|---|---|
| Real choir pairs, Choral Singing Dataset (48 pairs) | 39.9 ms mean 14.9 ms median |
44.8 ms mean 20.0 ms median (MERT 95M + mel) |
52.0 ms mean 34.8 ms median |
| Exact truth studio suite (120 cases, incl. harmony and octave stacks) | 3.2 ms mean octave stacks about 3.9 ms |
Not the main comparison | 39.5 ms mean |
| Held out Dagstuhl ChoirSet (12 pairs, run once, never tuned on) | 43.0 ms mean | 42.6 ms (HuBERT base) 45.2 ms (MERT 330M) 47.8 ms (MERT 95M) |
65.1 ms mean |
The held out test: a tie for first
The Dagstuhl ChoirSet run is the one we trust most, because we locked the engine first and ran it exactly once. RysUpAlign 2.1 scored 43.0 ms. HuBERT base scored 42.6 ms. That gap is inside the noise, so we call it a statistical tie for first place.
Both MERT models, the music research models we tested, landed behind RysUpAlign on that set. Doing nothing scored 65.1 ms, so every real aligner earned its keep there.
How we tested
No public benchmark existed for vocal to vocal alignment. Lyrics alignment benchmarks, like the MIREX lyrics tasks, measure something else: where words land against text. Lining up a double against a guide vocal is a different job. So we built one and called it RAB, the RysUp Align Bench.
- Real singers, real truth. 48 unison pairs from the public Choral Singing Dataset. Truth comes from its human corrected pitch tracks for each singer, which gives 5,365 matched note events.
- Exact truth in the studio. 120 cases built from real backing vocal and gang takes. We cut each double at syllables and slid the pieces by known amounts, so the correct answer is known to the sample. Cases cover typical slop, sloppy takes, harmonies and octave stacks.
- A held out set. 12 bass section pairs from the Dagstuhl ChoirSet, scored once at the very end.
- Same scorer for everyone. Every aligner, including ours, went through the same scoring code on the same audio.
What we did not test: commercial aligners. We could not run them through the benchmark, so we make no claim about beating any commercial plugin. The benchmark is public. Anyone who owns one can run it.
The research models we compared against are MERT (95M and 330M), HuBERT base, and several larger speech models on subsets. We gave them a fair shot by pairing them with the same kind of log mel branch our engine uses.
The open benchmark
RAB lives on GitHub at rysupaudio-lab/rysupalign-bench. Use it to reproduce our numbers or test your own aligner.
We also found and documented a truth problem while building it: one annotation file in the Choral Singing Dataset is a copy of another singer's track. We kept the scoring unchanged and document it with the results rather than quietly dropping the cases.
What is new in RysUpAlign 2.1
A learned alignment model
2.1 listens with a frozen HuBERT base encoder plus a 1.66 million parameter Rys Up alignment head, joined with a log mel branch. We trained the head on our own vocal recordings and the VocalSet dataset. It runs locally inside the plugin. No audio is uploaded.
A dense, pitch aware timing engine
The old engine picked a handful of confident anchors and threw the rest away. The new one matches the whole performance frame by frame and uses pitch movement to lock note changes, which is where most timing error lives. Harmony and octave stacks scored about 3 to 4 ms on the exact truth suite.
The plugin is the visualizer
- No Align button. Capture your vocals and RysUpAlign aligns them automatically.
- Per lane Timing, Pitch, Vibrato and Level. Dial each double on its own, right on its lane.
- Playback inside the plugin with solo and mute on every lane, so you can hear one double against the guide.
- Several guides for different sections, like a verse guide and a hook guide.
- Right click overrides. Sync a note, protect a phrase, or skip a phrase you want left alone.
- Zoom in when you need to check a single word.
Get RysUpAlign 2.1
RysUpAlign is $49.99 to own and ships AU, VST3 and AAX for Mac and Windows. It is also included in RysUpSuite with the rest of the paid lineup.
Already own it? Open RysUpHub or grab 2.1.0 from the installer hub.
Comparing options for stacks? Our vocal alignment plugin guide covers the workflow side, and the full lineup shows every Rys Up plugin. Questions about the benchmark or licensing the model? Reach us through the contact page.
Sources and data, October 8, 2026: RAB benchmark results for RysUpAlign 2.1.0 (released October 8, 2026); Choral Singing Dataset; Dagstuhl ChoirSet (CC BY 4.0); RysUpAlign update feed. Errors are mean or median absolute timing error over matched note onsets and note changes. The held out result is a tie within noise, not a clear win.