Engineering blog ·

Asking a notation program what it actually read

StaveWave reads sheet music and writes MusicXML, the file format notation programs use to swap scores (1). A file our own scorer likes is only half the job. The other half is what happens when a musician opens it in the program they already use.

We took a batch of our output files, opened each one, exported it back to MusicXML, and compared what went in with what came out. We run the same trip on our reference transcriptions (hand-checked answer-key files for the same scores) as a control. Anything the program does to the reference files is the program's habit, not our bug.

The round trip said we were losing marks

The first comparison looked bad. Slurs (curved lines that tell a player to connect notes smoothly), ties (curves that join two notes of the same pitch into one longer note) and hairpins (the wedge-shaped crescendo and diminuendo signs) were missing from the exported files, and more of them went missing from ours than from the reference files.

A round trip tests two steps at once: reading the file in and writing it back out. A missing slur in the output could mean the program never understood it. It could also mean the program understood it fine and dropped it on the way out. Comparing the two ends of the trip can't tell which.

So we asked the program

Most notation programs can run plug-ins, small scripts that walk through the score the program holds in memory. We wrote one that runs right after a file opens, before any layout or style step touches it. For each staff it logs how many notes, ties, slurs and hairpins the program now holds, which articulation marks are attached to notes, and where every slur starts and ends.

That gives a third view, between what we wrote and what the program exported: what it actually read.

Figure 1. The round trip compares our file with the program's export, so it tests the import and the export together. A plug-in that logs the score right after import measures the import step alone.
Figure 1. The round trip compares our file with the program's export, so it tests the import and the export together. A plug-in that logs the score right after import measures the import step alone.

Two checks came before trusting it. First, the note count on each staff had to match the file we fed in, or we couldn't line the staves up. It matched on nearly every file, and the few that didn't were compared part by part instead of being forced. Second, the plug-in had to read the reference files perfectly, since nothing in them is in doubt.

It did. Every slur, tie and hairpin in the reference files was there after import. On our files every tie and hairpin arrived, and all but a handful of slurs.

So most of what looked like our loss happened on the way out. The reference files lose the same marks on the same trip, so the loss sits in the program's export. A list of problems we had lined up to fix mostly went away. That is the cheapest kind of fix there is.

The handful that were real

What was left was small and specific: a slur that starts and ends on the same note, a slur that starts and never ends, and staccato dots on a percussion staff. Each is something we write that a careful reader could argue about, and the program settles the argument its own way.

The most useful case was a slur that came out too long.

Two slurs, one number

Piano and choral music often put two independent lines on one staff. Engravers call them voices. Each voice can carry its own slur, and the two slurs can overlap in time. MusicXML handles this with a number on each slur. The standard says the number distinguishes "multiple slurs when they overlap in MusicXML document order" (2). A stop marked 1 belongs to a start marked 1.

Some of our files gave two overlapping slurs the same number. Our own scorer never noticed, because it pairs slurs by voice. The import log showed what the program did with it. With two starts open under one number, it has to guess which one each stop closes. On one score it guessed wrong, and a slur that should have covered two bars ran on for more than twenty.

Figure 2. Two voices on one staff, each with a slur, both numbered 1. Written in file order, the starts and stops can be paired the way the composer meant (top) or another way (bottom), where one slur stays open until a much later stop.
Figure 2. Two voices on one staff, each with a slur, both numbered 1. Written in file order, the starts and stops can be paired the way the composer meant (top) or another way (bottom), where one slur stays open until a much later stop.

Counting slurs would never find this. The count is the same either way. You only see it by comparing where each slur starts and ends, which is exactly what the plug-in logs.

The log became the test

We used the same log to judge a change to how our writer numbers overlapping slurs. Passing meant three things. The too-long slurs come out at their length in the reference. No slur that already matched the reference moves. Nothing new goes wrong in a wider check across our test sets.

The first version failed. It fixed the long slurs and broke a different one: a slur that one of our recognizers writes twice in a row on the same notes, which the program had been reading correctly. It took three versions to fix the wrong slurs and leave every right one alone. Judged by slur counts, the first version would have looked like a clean win.

What this means if you convert a score

If you open a converted score and a slur or hairpin is missing, it may never have been missing from the file. Import, layout and export are separate steps, and any of them can drop a mark. The test that matters is the one closest to the musician: what the program shows right after it opens the file. That is the step we now measure.