The Systemic Failure of MIDI as a Musical Standard

[William Brookfield]

The Systemic Failure of MIDI as a Musical Standard


Consider, if you will, the "zzing" sound of the violin or the "shoop" sound of a foot close high hat. In the case of the violin, the pre-note sound is the "zz," is completely missing from the MIDI protocol. In the case of the foot close high hat, the pre-note sound "sh," is completely missing from the MIDI protocol. The "ing" sound on the downbeat represents the MIDI note-on and the release of the "g" sound is the MIDI note-off. What is missing from MIDI is the "zzz's" and the "sh's" the pre-note components of natural instruments. As a result MIDI is only good for sharp-attack techno sounds in which pre-note elements don't exist. The musical piece "Popcorn" is a good representation of this type of pre-note-less inorganic sound.

By failing to represent organic music, MIDI falls short of a truly musical interface. Musical sounds that include significant pre-note content will always present late in quantized ("corrected") midi recordings and live performances on midi synths will always (to one extent or another) distance the performer from the sounds being performed. There is a difference between sounds being performed and sounds being played back. There is a difference between playing an instrument and playing the radio.

The simplest solution as I see it, is to add some kind of "pre-note-on" data to the midi protocol and to build midi synths and controllers that start triggering (pre-note data) from the moment the "hammer leaves the gate." While I can see a number of technical solutions to this issue I am convinced that the most effective solution must start with an addition to the MIDI protocol itself -- to which instrument companies would then follow.

 

[happyrat1]

Actually it's dependent on the individual wave data for each note. ADSR controls can be applied to each note to minimize or eliminate prenote sounds or attack can be adjusted to the point where it becomes significant or even exaggerated.

Your argument is to add another data byte to the MIDI stream further increasing lag and complicating processing while a simpler solution is to ad a few microseconds of prenote data to the DSP processing the digitized waveform.

MIDI is still a remarkably robust standard and still serves us pretty well almost 4 decades after its inception.

Everyone wants to see a MIDI 3.0 standard come to pass but there is astonishingly little agreement as to what should go in it.

I've been participating in a discussion over on a Kurzweil forum about how the GM bank was intentionally omitted from the new PC4 models that just came out.

My standpoint is that it provides a minimum baseline for exchange of digital music files for collaborations and software designers alike and therefore should remain in service until it is replaced by something better. Not simply scrapped entirely to make room for yet another drum kit or bazouki sample.

Sure there are the XG and GS standards from Yamaha and Roland but they are proprietary and not de facto standards.

If there ever is a de facto MIDI 3.0 standard then to my mind it should specify a minimal soundfont which could easily be implemented across the spectrum of musical and computer devices.

Gary

 

[anotherscott]

This does not sound like a MIDI issue at all. There's no reason a standard MIDI-triggered sample of a note (or a synth programmed one) cannot include "pre" sounds. The problem is, how does the keyboard know what you want, when you want it? It can't add something to a note before you strike it. So for live performance, you'd have to play ahead of the beat and anticipate when you want it to "land." It would be simpler for assembling sequences, though, because you could play the notes in and then (after the fact) move the On events to occur sooner.

If I understand correctly what it is you're trying to accomplish.

 

[anotherscott]

You could also make the "pre" sounds available as separate samples, and map them to other keys; then play those keys essentially as "grace notes" in anticipation of the key you ultimately want to trigger. (Again, though, this is not a MIDI issue or something that calls for a MIDI solution.)

 

[Docbop]

I think you're talking more about sampling and who and how the samples were made. MIDI is basically just the messenger delivering data or collecting data sent. To me blaming MIDI is like blaming the microphone for a singer being bad.

 

[b3maniac]

I don't believe I have ever heard the term "MIDI" used in the context of a "musical standard." In your post you are making reference to the audio data which was not captured in sound sample. That is technical issue with how a sound sample was taken and processed. MIDI is a technical standard that describes a communications protocol, digital interface, and electrical connectors that connect a wide variety of electronic musical instruments, computers, and related audio devices for playing, editing and recording music. I remember the standard being expressed in early literature as a manufacturing standard for the electronic instrument industry. I propose that the level of organic audio detail referenced in the original post is exactly why real musicians and music recording still remains relevent in today's world. MIDI can't do everything. LOL

 

[Synthuaxed]

Agree with all of the above. If the sample includes the complete sound, including all of its components, then all that's required is you understand what the sound entails and time your playing accordingly.

 

[crlMIDI]

As an amateur in this field, I have the impression that MIDI is slowly disappearing from the scene because the standard is much to slow to adapt to requirements. The long-standing decline is obvious if you look at facilities for keyboard microtuning and temperaments, a minority but important interest. According to their way of working (1992?), though you can microtune a note in the sharp direction, but you can't flatten it without changing the note number down a semitone. That's totally useless because temperaments that were in use well into the 19th century can have two notes less than a semitone apart. We should not have to do ridiculously complicated programming to work around that.

I wish I knew what goes on in the minds of those who decide these MIDI matters.

Returning to the original question, unless MIDI can do its job of telling the instrument exactly what to do (though not how to do it), it would be best to let it die of old age.

 

[happyrat1]

Professionals in the industry still work with 5 pin MIDI every day. In fact many, myself included, refuse to buy any gear that does not ship with MIDI IN/OUT/Thru ports at all.

Low end budget gear aimed at consumers and their kids have pretty much dropped it in favor of cheaper USB connections but believe me, in pro audio MIDI is alive and well.

Gary

 

[Biggles]

If MIDI was to be laid to rest precisely what would it be replaced with?

 

[Synthuaxed]

Until MIDI can tie my shoes and brush my teeth for me, what good can it possibly be?

Guess I'll just keep whining about it until "they" make it do everything I can think of.

 

[anotherscott]

Also remember MIDI has nothing to do with audio per se. It is a standard way of sending and receiving data/instructions. And the last thing I'd want would be for it to go away. MIDI is the reason I can directly connect any 2019 keyboard with a MIDI port to a 1983 keyboard with a MIDI port and they will communicate. Sometimes simplicity is brilliance. MIDI can be extended with more functionality, but that doesn't mean doing away with what it already does.

Also, MIDI has never stopped any keyboard manufacturer from offering different 12-note per octave temperaments... MIDI has nothing to do with that. Microtuning with more than 12 notes in its repeating sequence is more complicated, but MIDI added sysex support for that in the 90s. It's covered in the last section of the article at https://www.midi.org/articles-old/microtuning-and-alternative-intonation-systems

There will be more enhancements to MIDI (2.0 has been around the corner for some time now ;-) ) but there's no reason to lose compatibility with everything that has come before.

 

[William Brookfield]

The problem as I see it is that MIDI is a representative protocol. MIDI “note-on” however does not always represent the sample's (WAV) internal note-on”/pulse/downbeat/transient peak. MIDI “note-on” instead represents the non-musical, sample start time (time zero) and not the samples internal (WAV) note-on time. Midi therefore suffers from a systemic failure of musical representation.

The solution, as I see, it is to augment the present standard/protocol with a pre-note MIDI data type for triggering the pre-note (wav) samples (in cases where pre-note material is required). Such cases include orchestral (two handed) “crrash" cymbals in which the “crr..” is the pre-note, the “..aa..” is the downbeat/note-on (the transient peak/pulse) and the “ssh” is the ring out. Other examples include violins, woodwinds, foot-close high hats etc.

Missing from this list are percussive (short attack) instruments such as pianos, organs, tom toms, snares etc. This is because the time (distance) between the between the sample start-time (MIDI note-on) and transient peaks (WAV note-on) for these instruments is so short that it can be safely ignored.

The Seaboard example is interesting but is very expensive in terms of the data stream – 16 midi channels per instrument. I would have to employ a separate MIDI system and my present system would still have its systemic issues. I also perform the drums with my feet.

I am not recommending that musicians learn to perform in countless difference wonky ways with various fingers in order to compensate for countless different pre-note latencies. I am recommending instead, a comprehensive system-based solution that permits musicians to perform in the coherent manner to which they were trained.

The Seaboard Rise applies a lot of data to post note-on processing but does nothing to address the pre-note issue addressed here...

Or perhaps I am mistaken.. How would the Seaboard fix the following problem...

I wish to perform live in my band with three layers. Piano on midi channel 1, a string patch on midi channel 2 and an orchestral crash (on midi channel 3) on the high velocity notes taken from control channel MIDI 1. The problem is that piano being a "ding" sound has almost no pre-note content but violins and orchestral crashes have significant pre-note content and subsequently they always come in late compared to the piano. What are the possible live performance solutions here?

(A) Chop off the pre-note content of the violins and orchestral crashes to make them more piano like in their attack (this destroys the natural sound of these instruments) - not good

(B) Delay the piano to bring it into time with the late violins and possibly the crash. This makes the piano delayed and subsequently most awkward to play and nearly impossible to groove with the band, moreover the crash pre-note length and the violin pre-note length may be of different lengths. (In the case of three or more different pre-note lengths, live synchronization is physically impossible).

None of these are real solutions. In the first case the sounds are damaged and in the second case the performance interface is damaged. I am unable to see how the Seaboard solves this issue. The problem here is that midi note-on does not correspond to sample note-on but instead applies to the sample playback start-time zero.

What is needed is an extended midi standard that requires that a midi note-on must correspond (in fixed milliseconds) to the sample note-on. Because some samples have significant pre-note content this extra sound must {according to this standard} be allocated to the pre-note sample and triggered by midi pre-note-on as the key (piano hammer) is leaving the resting position and heading for the later note-on.

In order to provide performance timing flexibility the pre-note would typically include a looped element - just as the note-on would typically contain a looped portion.

As a professional live MIDI musician I am acutely aware of the healing and uplifting power of music. I therefore take music and musical interfaces very seriously. The pre-note problem has made me acutely aware of the musical limitations of MIDI and my inability to work well with various musical sounds and styles (jazz and classical music are two examples). In 1999 I was commissioned to produce a MIDI classical symphony. This work however was like "herding cats" due to the countless pre-note articulations of countless orchestral samples.

I do realize, however, that many people use synth type keys for MIDI and that these interfaces provide very little pre-note room. This is due to there being very little distance between the synth keys resting position and the keys depressed note-on position. This is not true for piano keys (or pk5 keys) wherein significant pre-note room exists. A MIDI pre-note protocol would in no way conflict with the pianists training, whereas any requirement that certain specific pianist fingers must land early or late to compensate for pre-note content would indeed conflict with previous training.

Due to the pre-note content of the foot closed high hat, I am presently forced to perform in a hobbled compensatory manner on stage while performing on my (Roland PK5) foot drums.

In the case of the acoustic drum kit, the foot close high hat serves as a pre-note signal (both audio and visual) for other band members that a down beat is about to occur. The midi protocol however, being devoid of midi pre-note signalling makes real-time grooving (locking into) the rhythm of a midi (Roland Pk5) drummer such as myself significantly more difficult for other band members. The addition of midi pre-note would solve this problem for live performances, providing both, anticipatory audio and anticipatory visual pre-note data to all band members.

In order to remedy these issues am now seeking a trigger mechanism of some kind that I could attach to my MIDI pedals (Roland PK-5's) that could detect when the pedal leaves the resting position. I,E, I am looking to "Macgiver" or "Jerry rig" my system in order to compensate for systemic representational failure of the MIDI protocol. If anyone is aware of a third party triggering mechanism that would work for this please let me know.

In conclusion. I am not recommending that musicians learn to perform in countless different wonky ways in order to compensate for countless different pre-note latencies. I am recommending instead, a comprehensive, standardized, system-based solution that permits musicians to perform in the manner to which they are trained.

Because the industry may well be unresponsive to this issue, I am presently looking for small aftermarket sensors that could be added to my present (PK5) midi rig. These would detect the initial movement of the keys from their resting position and instantly send a midi signal. (It doesn't matter what type midi of signal because I can always translate midi of any type into midi note-on and visual data).

PS..It turns out that my wife's brother's son is technically savvy and he is recommending certain small industrial sensors mounted above my PK5 keys, coupled with an Arduino circuit board to produce MIDI "pre-note" data. Stay tuned...


*(In orchestral crashes, two cymbals are held in two hands and then pushed together. As with the foots close high-hat, the pre-note sound is produced as the cymbals connect only partially at first {pre-note} and then fully {midi note on} at the down beat ).

 

[anotherscott]

The problem as I see it is that MIDI is a representative protocol. MIDI “note-on” however does not always represent the sample's (WAV) internal note-on”/pulse/downbeat/transient peak. MIDI “note-on” instead represents the non-musical, sample start time (time zero) and not the samples internal (WAV) note-on time.

No... MIDI "note on" represents the point at which you've pressed the key enough to trigger a particular sensor. What the software does with that event is entirely up to the programmer. It can play a sample immediately, or after some times, or a sequence of two or more samples, or sounds that aren't generated by samples at all. What it can't do is go back in time and play a sound before you triggered the Note On event.

So as I said above, for what you want, your choices are to build a "pre" sound into the samples (which you could certainly do, but you'd have to play ahead of the beat in a carefully timed fashion), or assign the "pre" sound samples to other keys, and play those keys as grace notes to the main tone you want to sound.

 

[picman]

"... you'd have to play ahead of the beat in a carefully timed fashion), or assign the "pre" sound samples to other keys, and play those keys as grace notes to the main tone you want to sound "

which is exactly:

"...musicians learn to perform in countless different wonky ways in order to compensate for countless different pre-note latencies "

 

[William Brookfield]

Well, I built a working prototype.. and it works great!

see..

and..

and..

 

[picman]

Well, I built a working prototype.. and it works great!

Ingenious, but it is essentially a hardware solution, is it not?
As I see it, the problem you describe in a live performance setting is not with MIDI but with the instrument used; trying to imitate a foot closed high hat or a violin with a keyboard.
Not that it can't be done, because it can - with appropriate sound samples. The pre-note sound would be simply part of the sample. In which case the keyboard player would have to hit the keys somewhat ahead of time, just as the violin player starts to draw the bow over the string slightly ahead of time in order to reach peak loudness on the beat. With velocity sensitive keys, triggering or not triggering the sample that includes the pre-note sound would also be possible. A specially made keyboard wouldn't even be required, because this decision could be made dynamically in software (based on how quickly the key was pressed down).

Correct me if I'm wrong, but I don't think MIDI 'note-on' was ever mandated to be peak loudness or even 'right on the beat'.

This is not to belittle the work you've done, because I think it is great. However, I don't think it is a fix for a (perceived) shortcoming of the MIDI spec.

Just my 2 cents worth.

 

[anotherscott]

As I see it, the problem you describe in a live performance setting is not with MIDI
...
Not that it can't be done, because it can - with appropriate sound samples. The pre-note sound would be simply part of the sample. In which case the keyboard player would have to hit the keys somewhat ahead of time
...
Correct me if I'm wrong, but I don't think MIDI 'note-on' was ever mandated to be peak loudness or even 'right on the beat'.

Yes, this is also what I was talking about earlier in the thread.

 

[crlMIDI]

Also, MIDI has never stopped any keyboard manufacturer from offering different 12-note per octave temperaments... MIDI has nothing to do with that. Microtuning with more than 12 notes in its repeating sequence is more complicated, but MIDI added sysex support for that in the 90s. It's covered in the last section of the article at https://www.midi.org/articles-old/microtuning-and-alternative-intonation-systems

Keyboard manufacturers do offer different 12-note per octave temperaments, but, as has been pointed out elsewhere, they make sure you can't write transportable code to make your own modifications. Instruments like the Roland C30 harpsichord and the FP 50 piano do have a few present temperaments. You can re-tune via sysex but it's difficult.

A while ago I posted code designed to let you tune a Roland JV or XP interactively http://www.music.chrblee.net/tuningaid/ . When my old JV stopped working, I discovered that subsequent models like the FP 50 have totally different structures for the relevant sysex commands. I know we should be programming in the object-oriented manner, but the class structures of objects you'd need for different equipment from this one manufacturer would be so different that in practice hardly any of the code could have been made transportable.

I mentioned above (3 years ago) that the midi tuning standard seems designed to aggravate the situation.

 

[SeaGtGruff]

The whole issue of scale tuning on a keyboard is deceptively deep-- and, to me at least, deeply interesting.

Not only do different manufacturers have different manufacturer-specific SysEx messages for adjusting the tuning of the notes of the chromatic scale, but different models from a given manufacturer might use different messages.

And some keyboards might not have any scale-tuning abilities at all, at least not per se, although if they respond to Pitch Bend messages then it might be possible-- if awkward-- to use different MIDI channels for notes which share the same offsets from standard 12TET tuning, and apply specific Pitch Bend offests to different channels.

As far as keyboards that do respond to scale-tuning messages, certain models have panel controls-- often in the form of buttons that look like a small 12-note keyboard-- which are dedicated to adjusting the tuning of each of the 12 notes of the chromatic scale. But those models are usually marketed toward parts of the world where the need for custom scale tuning is in higher demand, such as in Arabic or Hindu music.