NRPNs Part 1: The Basics

I started working on a Mopho/Tetra patch for Ableton Live now that I have Max for Live up and running. It's really a very cool level of integration, there's tons you can do with the two together! Of course since Ableton doesn't have fully featured MIDI handling you quickly run into some limitations...

For part 1 I'll give a little background for those who aren't intimately familiar with the MIDI specification:

Normally when you turn a knob or move a slider, you're changing a MIDI Control Change message (CC for short) which is the standard for controlling values on a synth. This is great for most things: since MIDI uses 7 bit numbers for data, you have up to 128 different parameters you can change. and each has 128 different values you can move it to (0 to 127). 128 is a good amount of parameters we can use, but the MIDI spec takes a handful of those and assigns them to predetermined or special functions so we lose almost 30 or so and we're down to 100 usable parameters in a standard setup.

100 parameters and each from 0 to 127... so what happens if we want to control more than 100 parameters or we have a parameter with a very large range that goes past 127? Lots of synths are coming out with more complicated controls and parameters and it's difficult to limit control to just 100 values. When we need more, we have a couple options:

1. Remapping existing values (commonly pitchbend and poly key pressure) - a pitchbend message has a resolution of 14 bits. That means we have a range of 0 to 16383, plenty for a high resolution message. However we only have one pitchbend message per channel, so we're limited to a maximum of 16 high resolution parameters this way. This works pretty well if you have a small number of parameters you need increased resolution for. The best example of this is in any Mackie control surface: Mackie uses pitchbend messages for each volume slider to increase the resolution. Poly key pressure has a range of only 0-127 but we have a pitch assigned to each one as well, so it gives us a possible 0-127 new parameters with a range of 0-127. Some DAWs and controllers don't support Poly key pressure though as it's a fairly uncommon feature in modern keyboard controllers.
2. Sysex - Sysex messages can have any number of bytes in them so we have tons of room to play around with things. We can have as many parameters and whatever resolution we desire! This is how it's done in the Evolver synths. The problem is since you can do anything with it each instrument normally has it's own standard for how it trasmits data. When you have your own standard you have to have some way to create the messages and interpret the messages. Most controllers and DAWs offer limited (if any) support for Sysex construction and deconstruction of values. You can use an interpreter like the powerful Bome's Midi Translator for Windows and Mac or a more simple free utility such as MIDIOX for Windows or MIDI Pipe for Mac OSX to do the translation. This is a lot of work to do, especially when you have to create a different mapping for every device you want to use, plus a lot of controllers and DAWs don't support Sysex at all making it impossible to set it up simply.
3. NRPN - The MIDI protocol's answer to needing higher resolution values is Non-registered Parameter Numbers. An NRPN is essentially 4 CC number and value pairs that are combined to give much higher resolution. You have a full range of 0 to 16383 for 16384 different parameters we can control. Perfect! Well almost...

To find out what the problems are we need to understand what exactly an NRPN is and how to use them. So let's start by breaking down what an NRPN message looks like in MIDI. Here's an NRPN message in raw MIDI bytes for changing NRPN #572 to a value of 163 on MIDI channel 1:

     B0 63 04 B0 62 3C B0 06 01 B0 38 23

When we look at a MIDI stream like this, to determine if we're seeing a command (which MIDI refers to as 'status') or data we see if the first bit in the byte is a 1. This means 80-FF are all command bytes and the start of a new MIDI command. In our stream, the only bytes we have in that range are the bytes 'B0'. Looking at the MIDI spec, a command byte has two components to it, the first 4 bits indicate what command we are running and the last 4 bits indicate which MIDI channel we're on. So let's divide 'B0' up into parts:

    B = first 4 bits = Control Change (CC)
    0 = last 4 bits = MIDI Channel 1

That means we're seeing 4 Control Change messages for one NRPN change. Now let's figure out what CCs are being changed. First we'll divide the stream into the 4 individual CC Packets:

    B0 63 04
    B0 62 3C
    B0 06 01
    B0 26 23

Each Control Change message has 2 bytes of data following it, the first byte telling us which CC# we are changing and the second byte telling us the value that we are changing to. Since all the values are in hexadecimal form we'll also convert them to decimal to put them in a more readable form. The easiest way of doing this is in google, add a 0x to the beginning of each number to indicate that it's in hex form and enter in google '0x63 to decimal' which returns the decimal value of 99. So all our packets, expanded:

    Packet             Hex                     Decimal
    B0 63 04 - CC 0x63 = 0x04 -> Set CC# 99 to 4
    B0 62 3C - CC 0x62 = 0x3C -> Set CC# 98 to 60
    B0 06 01 - CC 0x06 = 0x01 -> Set CC# 6 to 1
    B0 26 23 - CC 0x26 = 0x23 -> Set CC# 38 to 35

So now we know exactly what it looks like when an NRPN is changed, but it's still pretty confusing... what do each of the 4 numbers represent? The first thing we need to figure out is exactly what those CC#s mean. Each of the 4 CC#s we see are assigned by the MIDI spec to special values for high precision changes. Going to the spec (A good list of MIDI messages and what they mean is here: http://www.midi.org/techspecs/midimessages.php) we find:

    CC# 99 (0x63) -  Non-registered Parameter Number Most Significant Byte (NRPN MSB)
    CC# 98 (0x62) -  Non-registered Parameter Number Least Significant Byte (NRPN LSB)
    CC# 6   (0x06) -  Data Entry Most Significant Byte (Data MSB)
    CC# 38 (0x26) -  Data Entry Least Significant Byte (Data LSB)

    Note: Sometimes documents will refer to MSB as 'Coarse' and LSB as 'Fine' for the Data Entry values

Looking at the names, it looks like we only really have 2 different parameters: NRPN and Data. Since each MIDI CC message has a maximum of 7 bits of data and an NRPN has 14 bits of data, we need to combine these messages to form the NRPN change. Let's go back to the data part of those messages now...

Remember how we found out which bytes were a command? We looked at the first bit to see if it was a 1. With a data packet, the first bit is always a 0. Since a MIDI packet is always 8 bits long, that means we lose 1 bit in every data packet to tell us whether or not it's a status or data byte, leaving us with 7 bits of useful data. So we can throw that first bit out. Now we have 7 bits + 7 bits = 14 bits, perfect! When we add them together the order of the bits will be MSB (Coarse changes) then LSB (Fine changes). To add them together we need to know what the 7 bits actually looks like for each value. We'll use the hex we found from the raw stream (0x##) and convert to binary (0b#### ####). The same trick with google can be used here, just enter '0x04 to binary':

    In Hex: CC# 0x63 to 0x04 ->  0x04 in binary is 0b0000 0100 so we take out the first bit and we end up with: 0x04 = 0b 000 0100

    NRPN MSB = 0x04 = 0b 000 0100
    NRPN LSB = 0x3C = 0b 011 1100
    Data MSB = 0x01 = 0b 000 0001
    Data LSB = 0x23 = 0b 010 0011

Now to glue them together we just write the 7 bits of the MSB then the 7 bits of the LSB to get the final 14 bit values:

    NRPN = 0b 00 0010 0011 1100 = 0x023C = 572
    Data = 0b 00 0000 1010 0011 = 0x00A3 = 163

At this point we know basically what's going on with NRPNs and what it looks like each time you send an update over MIDI. Next it's time to do something interesting now that we know what's going on.