Rhythmical Staff Notation
May 24, 2020
In this post, I want to find out how I can render rhythmical format as staff notation.
About rhythmical
In short, rhythmical is a small lib that transforms a nested array of primitives (typically note strings) to a flat array of note events:
Rhythm.render(['C', ['E', 'G'], 'B', 'D'], 4).map(({ value, time, duration }) => [value, time, duration]);
outputs:
[
['C', 0, 1],
['E', 1, 0.5],
['G', 1.5, 0.5],
['B', 2, 1],
['D', 3, 1],
];
That flat output can easily be played back with Tone.js or anything that works like MIDI.
In my opinion, this is the closest to human way to represent rhythm as a data structure + it is also really easy to understand. The format is strongly influenced by TidalCycles.
To find out more, you can either read the README or check out the (still experimental) REPL.
Staff Rendering in the Browser
As staff notation is the standard way to read and write music among musicians, i would be very happy if i could render rhythmical directly as a score. In the digital world, musicXML is the broadly supported standard format.
In the browser, one of the most feature rich libs to render scores is vexflow, though it is relatively low level. With opensheetmusicdisplay musicXML can be rendered directly with vexflow at its core.
To bridge the gap between musicXML and a hackable solution, I started implementing a json2musicXML renderer. As musicXML is really verbose and XML is not the best format to parse with JS, I wanted to be able to generate a valid musicXML file from JSON. Also, musicXML is relatively low level, as it is non opinionated about how accidentals, beams and ties should be used.
Unwritten "Rules" of Sheet Music Notation
There is a relatively clear consensus among musicians of how a good music sheet should look like, so most of the decisions could be made automatically (with json2musicxml), for example:
- accidentals should only appear once in a measure
- All successive notes that are shorter than a quarter (8ths, 16ths etc..) should be beamed together
- The beams should not cross barlines, or the middle of a bar
- Also, note values should not cross the middle of a bar, split with a tie
- The bars should be grouped logically (e.g. 4 bars per line)
Most of those decisions benefit readability. These "rules" are more or less automatically applied by every good music notation program.
What's missing
If all of the above rules are applied automatically, only a few things are needed to render a score:
- clef
- time signature
- list of notes with
- note (scientific pitch notation)
- duration
- tie flag
That's it! So a pseudo JSON format as input could look like:
{
clef: 'treble',
timeSignature: '4/4',
measures: [
[['C4', 4], ['D4', 8], ['E4', 8], ['F4', 4], ['G3', 8], ['C4', 8]],
[['C4', 1, true]]
]
}
Dont fixate too much about the actual format, this could also be an object syntax or slightly different semantics. Of course there are several more advanced score notation features like multiple voices and multiple staffs, jump signs chord symbols etc. but for this post, I will just concentrate on the core transformation of rhythmical to measures.
The goal of this post
This post does not aim to present the last solution, but more of a proof of concept, with a basic way to transform rhythmical to a format like the above. In the end, I probably will go with json2musicxml but the transition from rhythmical will be usable nevertheless.
Score notation vs rhythmical
I already did a basic implementation of a score rendering component using vexflow. For example, this is the classic tetris melody:
show source
<Score
width={600}
height={100}
staves={[
{
setBegBarType: 'REPEAT_BEGIN',
notes: [
['e5', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['c5', 8],
['b4', 8]
]
},
[
['a4', 'q'],
['a4', 8],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', '8r'],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
]
]}
/>
<Score
width={600}
height={100}
timeSignature=""
clef=""
staves={[
[
['b4', '8r'],
['d5', 8],
['b4', '8r'],
['f5', 8],
['a5', 'q'],
['g5', 8],
['f5', 8]
],
[
['e5', 'q'],
['b4', '8r'],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
{
notes: [
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
],
setEndBarType: 'REPEAT_END'
}
]}
/>
See this page for more examples on the score component. There are still a few features missing, like tuplets or automated accidentals, but it is enough for a proof of concept.
In rhythmical format (non DSL), the above score can be represented with:
let events = Rhythm.render(
[
['e5', ['b4', 'c5'], 'd5', ['c5', 'b4']],
['a4', ['a4', 'c5'], 'e5', ['d5', 'c5']],
['b4', ['r', 'c5'], 'd5', 'e5'],
['c5', 'a4', 'a4', 'r'],
[['r', 'd5'], ['r', 'f5'], 'a5', ['g5', 'f5']],
['e5', ['r', 'c5'], 'e5', ['d5', 'c5']],
['b4', ['b4', 'c5'], 'd5', 'e5'],
['c5', 'a4', 'a4', 'r'],
],
8
).map(({ value, time, duration }) => [value, time, duration]); // simplify
show output
[
['e5', 0, 0.25],
['b4', 0.25, 0.125],
['c5', 0.375, 0.125],
['d5', 0.5, 0.25],
['c5', 0.75, 0.125],
['b4', 0.875, 0.125],
['a4', 1, 0.25],
['a4', 1.25, 0.125],
['c5', 1.375, 0.125],
['e5', 1.5, 0.25],
['d5', 1.75, 0.125],
['c5', 1.875, 0.125],
['b4', 2, 0.25],
['r', 2.25, 0.125],
['c5', 2.375, 0.125],
['d5', 2.5, 0.25],
['e5', 2.75, 0.25],
['c5', 3, 0.25],
['a4', 3.25, 0.25],
['a4', 3.5, 0.25],
['r', 3.75, 0.25],
['r', 4, 0.125],
['d5', 4.125, 0.125],
['r', 4.25, 0.125],
['f5', 4.375, 0.125],
['a5', 4.5, 0.25],
['g5', 4.75, 0.125],
['f5', 4.875, 0.125],
['e5', 5, 0.25],
['r', 5.25, 0.125],
['c5', 5.375, 0.125],
['e5', 5.5, 0.25],
['d5', 5.75, 0.125],
['c5', 5.875, 0.125],
['b4', 6, 0.25],
['b4', 6.25, 0.125],
['c5', 6.375, 0.125],
['d5', 6.5, 0.25],
['e5', 6.75, 0.25],
['c5', 7, 0.25],
['a4', 7.25, 0.25],
['a4', 7.5, 0.25],
['r', 7.75, 0.25],
];
The output format is [note, time, duration].
We can easily calculate the note value with 1/duration + floor the time to get the bar index (this works because we rendered 8 bars to 8 seconds):
events.map(([note, time, duration]) => [note, Math.floor(time), 1 / duration);
show output
[
['e5', 0, 4],
['b4', 0, 8],
['c5', 0, 8],
['d5', 0, 4],
['c5', 0, 8],
['b4', 0, 8],
['a4', 1, 4],
['a4', 1, 8],
['c5', 1, 8],
['e5', 1, 4],
['d5', 1, 8],
['c5', 1, 8],
['b4', 2, 4],
['r', 2, 8],
['c5', 2, 8],
['d5', 2, 4],
['e5', 2, 4],
['c5', 3, 4],
['a4', 3, 4],
['a4', 3, 4],
['r', 3, 4],
['r', 4, 8],
['d5', 4, 8],
['r', 4, 8],
['f5', 4, 8],
['a5', 4, 4],
['g5', 4, 8],
['f5', 4, 8],
['e5', 5, 4],
['r', 5, 8],
['c5', 5, 8],
['e5', 5, 4],
['d5', 5, 8],
['c5', 5, 8],
['b4', 6, 4],
['b4', 6, 8],
['c5', 6, 8],
['d5', 6, 4],
['e5', 6, 4],
['c5', 7, 4],
['a4', 7, 4],
['a4', 7, 4],
['r', 7, 4],
];
Now, the format is [note, bar, value].
This is pretty close to the score format. The last step is to group the notes by bar and apply some (vexflow specific) fixes for rests and quarter notes:
events.reduce((groups, [note, bar, value]) => {
if (!groups.length || bar > groups.length - 1) {
groups.push([]);
}
if (value === 4) {
value = 'q'; // this fixes a vexflow bug: value 4 has wrong stem directions, q works
}
if (note === 'r') {
// fix rests
value = value + 'r';
note = 'b4';
}
groups[groups.length - 1].push([note, value]);
return groups;
}, []);
show output
[
[
['e5', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['c5', 8],
['b4', 8],
],
[
['a4', 'q'],
['a4', 8],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8],
],
[
['b4', 'q'],
['b4', '8r'],
['c5', 8],
['d5', 'q'],
['e5', 'q'],
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r'],
],
[
['b4', '8r'],
['d5', 8],
['b4', '8r'],
['f5', 8],
['a5', 'q'],
['g5', 8],
['f5', 8],
],
[
['e5', 'q'],
['b4', '8r'],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8],
],
[
['b4', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['e5', 'q'],
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r'],
],
];
This is it! We can test if it really works by throwing this into the Score component:
show source
<Score
width={600}
height={100}
staves={[
[
['e5', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['c5', 8],
['b4', 8]
],
[
['a4', 'q'],
['a4', 8],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', '8r'],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
]
]}
/>
<Score
width={600}
height={100}
timeSignature=""
clef=""
staves={[
[
['e5', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['c5', 8],
['b4', 8]
],
[
['a4', 'q'],
['a4', 8],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', '8r'],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
]
]}
/>
Awesome! We can now sum everything up with this function:
import { Rhythm } from 'rhythmical';
import { NestedRhythm } from 'rhythmical/lib/Rhythm';
export function rhythmicalScore(rhythm: NestedRhythm<string>) {
return Rhythm.render(rhythm, rhythm.length)
.map((e) => [e.value, Math.floor(e.time), 1 / e.duration])
.reduce((groups: any[][], [note, bar, duration]) => {
if (!groups.length || bar > groups.length - 1) {
groups.push([]);
}
if (duration === 4) {
duration = 'q';
}
if (note === 'r') {
duration = duration + 'r';
note = 'b4';
}
groups[groups.length - 1].push([note, duration]);
return groups;
}, []);
}
This function is now usable with the score component directly:
show source
<Score
width={600}
height={100}
staves={rhythmicalScore([
['e5', ['b4', 'c5'], 'd5', ['c5', 'b4']],
['a4', ['a4', 'c5'], 'e5', ['d5', 'c5']],
['b4', ['r', 'c5'], 'd5', 'e5'],
['c5', 'a4', 'a4', 'r']
])}
/>
<Score
width={600}
height={100}
timeSignature=""
clef=""
staves={rhythmicalScore([
[['r', 'd5'], ['r', 'f5'], 'a5', ['g5', 'f5']],
['e5', ['r', 'c5'], 'e5', ['d5', 'c5']],
['b4', ['b4', 'c5'], 'd5', 'e5'],
['c5', 'a4', 'a4', 'r']
])}
/>
Now compare the source above to the non-rhythmical version:
show source
<Score
width={600}
height={100}
staves={[
[
['e5', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['c5', 8],
['b4', 8]
]
[
['a4', 'q'],
['a4', 8],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', '8r'],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
]
]}
/>
<Score
width={600}
height={100}
timeSignature=""
clef=""
staves={[
[
['b4', '8r'],
['d5', 8],
['b4', '8r'],
['f5', 8],
['a5', 'q'],
['g5', 8],
['f5', 8]
],
[
['e5', 'q'],
['b4', '8r'],
['c5', 8],
['e5', 'q'],
['d5', 8],
['c5', 8]
],
[
['b4', 'q'],
['b4', 8],
['c5', 8],
['d5', 'q'],
['e5', 'q']
],
[
['c5', 'q'],
['a4', 'q'],
['a4', 'q'],
['b4', '4r']
]
]}
/>
I think this is a pretty drastic improvement, as the lines went down from 73 to 22 which is 70% less. With this method, I can write music directly in MDX as part of my blog posts.
Of course this is still pretty basic as it does not support tuplets / ties / repeat signs etc, but it is a good starting point.
Next Steps
- colored notes to sync actual playback with score
- improve score component to accept multiple lines
- chords
- ties
- tuplets
- use extended rhythmical format for
- barlines
- multiple voices
- articulation
- etc...
- implement json2musicxml and use rhythmical with it