SDL_Mixer: Improved MIDI playback patch

SDL Development
1

I just submitted a patch at http://bugzilla.libsdl.org/show_bug.cgi?id=1267 that improves MIDI playback in two ways:

  1. It trims leading silence from the song.? Some MIDIs begin with a few seconds of silence, which can make working with them in a game awkward.
  2. Fixed the playback code to respond properly to Controller Event #111, the looping command.

The patch only applies to native_midi_win32.c, but the logic changes are simple enough that they could probably be ported to any of the other playback engines.

Mason

2

This doesn’t sound like a good idea. The silence might be there on
purpose. If you want to get rid of it, you would simply use a MIDI
editor to edit the MIDI file.On 08/07/2011 08:30 AM, Mason Wheeler wrote:

I just submitted a patch at
http://bugzilla.libsdl.org/show_bug.cgi?id=1267 that improves MIDI
playback in two ways:

  1. It trims leading silence from the song. Some MIDIs begin with a few
    seconds of silence, which can make working with them in a game awkward.
3

Exactly. The most common reason for the delay is that some synths (older ones
in particular - and samplers, obviously) need a moment to respond to patch
changes and other initialization events. Without the delay, the first few
notes may be skipped or delayed.On Sunday 07 August 2011, at 08.16.02, Nikos Chantziaras wrote:

On 08/07/2011 08:30 AM, Mason Wheeler wrote:

I just submitted a patch at
http://bugzilla.libsdl.org/show_bug.cgi?id=1267 that improves MIDI
playback in two ways:

  1. It trims leading silence from the song. Some MIDIs begin with a few
    seconds of silence, which can make working with them in a game awkward.

This doesn’t sound like a good idea. The silence might be there on
purpose. If you want to get rid of it, you would simply use a MIDI
editor to edit the MIDI file.

–
//David Olofson - Consultant, Developer, Artist, Open Source Advocate

.— Games, examples, libraries, scripting, sound, music, graphics —.
| http://consulting.olofson.net http://olofsonarcade.com |
’---------------------------------------------------------------------’

4
  1. It trims leading silence from the song. Some MIDIs begin with a few
    seconds of silence, which can make working with them in a game awkward.

This doesn’t sound like a good idea.? The silence might be there on
purpose.? If you want to get rid of it, you would simply use a MIDI
editor to edit the MIDI file.

Exactly. The most common reason for the delay is that some synths (older ones
in particular - and samplers, obviously) need a moment to respond to patch
changes and other initialization events. Without the delay, the first few
notes may be skipped or delayed.

How old are we talking here?? I added this functionality to fix an issue with MIDI
playback that I ran into while trying to clone a game engine that was originally
released back in 2000.? It automatically trims leading silence from MIDI files, and
apparently the programmers back then were confident that that wouldn’t screw
anything up, even on what was “older” systems to them back then.? So what are
we talking about here? Early 90s? 80s?? It’s kinda ridiculous to let obsolete
technology influence modern design decisions like that…>From: David Olofson

Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch
On Sunday 07 August 2011, at 08.16.02, Nikos Chantziaras wrote:

On 08/07/2011 08:30 AM, Mason Wheeler wrote:

5
  1. It trims leading silence from the song. Some MIDIs begin with a few
    seconds of silence, which can make working with them in a game
    awkward.

This doesn’t sound like a good idea. The silence might be there on
purpose. If you want to get rid of it, you would simply use a MIDI
editor to edit the MIDI file.

Exactly. The most common reason for the delay is that some synths (older
ones in particular - and samplers, obviously) need a moment to respond to
patch changes and other initialization events. Without the delay, the
first few notes may be skipped or delayed.

How old are we talking here? I added this functionality to fix an issue
with MIDI playback that I ran into while trying to clone a game engine
that was originally released back in 2000. It automatically trims leading
silence from MIDI files, and apparently the programmers back then were
confident that that wouldn’t screw anything up, even on what was "older"
systems to them back then. So what are we talking about here? Early 90s?
80s? It’s kinda ridiculous to let obsolete technology influence modern
design decisions like that…

Well, some late 90s and early 00s studio synths still had this issue - some
because of slow MCUs driving the DSPs, others apparently in a clumsy attempt
to guarantee click-free patch changes during performances. Haven’t really been
in the loop with hardware synths since.

Either way, the more relevant question is probably, are there still consumer
sound cards and OS/driver softsynths in use that have this problem?

(Even the now ancient SoundBlaster AWE32 would respond instantly, but then
again, it had no MCU that could fall behind while shuffling patch data. The
"DSP" was controlled directly by the driver running on the host CPU.)On Sunday 07 August 2011, at 15.00.45, Mason Wheeler wrote:

From: David Olofson <@David_Olofson>
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch
On Sunday 07 August 2011, at 08.16.02, Nikos Chantziaras wrote:

On 08/07/2011 08:30 AM, Mason Wheeler wrote:

–
//David Olofson - Consultant, Developer, Artist, Open Source Advocate

.— Games, examples, libraries, scripting, sound, music, graphics —.
| http://consulting.olofson.net http://olofsonarcade.com |
’---------------------------------------------------------------------’

6

How old are we talking here?? I added this functionality to fix an issue
with MIDI playback that I ran into while trying to clone a game engine
that was originally released back in 2000.? It automatically trims leading
silence from MIDI files, and apparently the programmers back then were
confident that that wouldn’t screw anything up, even on what was "older"
systems to them back then.? So what are we talking about here? Early 90s?
80s?? It’s kinda ridiculous to let obsolete technology influence modern
design decisions like that…

Well, some late 90s and early 00s studio synths still had this issue - some
because of slow MCUs driving the DSPs, others apparently in a clumsy attempt
to guarantee click-free patch changes during performances. Haven’t really been
in the loop with hardware synths since.

Either way, the more relevant question is probably, are there still consumer
sound cards and OS/driver softsynths in use that have this problem?

(Even the now ancient SoundBlaster AWE32 would respond instantly, but then
again, it had no MCU that could fall behind while shuffling patch data. The
"DSP" was controlled directly by the driver running on the host CPU.)

Exactly.? I don’t really think that the hardware that has trouble with this is the
same as the hardware being targeted by SDL apps.>From: David Olofson

Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch
On Sunday 07 August 2011, at 15.00.45, Mason Wheeler <@Mason_Wheeler> wrote:

7

How old are we talking here?? I added this functionality to fix an issue
with MIDI playback that I ran into while trying to clone a game engine
that was originally released back in 2000.? It automatically trims
leading
silence from MIDI files, and apparently the programmers back then were
confident that that wouldn’t screw anything up, even on what was "older"
systems to them back then.? So what are we talking about here? Early 90s?
80s?? It’s kinda ridiculous to let obsolete technology influence modern
design decisions like that…

Well, some late 90s and early 00s studio synths still had this issue - some
because of slow MCUs driving the DSPs, others apparently in a clumsy
attempt
to guarantee click-free patch changes during performances. Haven’t really
been
in the loop with hardware synths since.

Either way, the more relevant question is probably, are there still
consumer
sound cards and OS/driver softsynths in use that have this problem?

(Even the now ancient SoundBlaster AWE32 would respond instantly, but then
again, it had no MCU that could fall behind while shuffling patch data. The
"DSP" was controlled directly by the driver running on the host CPU.)

Exactly.? I don’t really think that the hardware that has trouble with this
is the
same as the hardware being targeted by SDL apps.

The hardware issue is just one of the problems with this.
If this patch is accepted and latter found a single case where this is
bad you end up with a situation where you can’t fix the library
without breaking compatibility.

This is one of those cases where it’s bad to put policy on a library.
It’s not up to the library to hard-code application behavior.
If it’s a common case (skipping silence) then add a flag for it, but
don’t do it on the default case.

Just an example where this is just bad: you implement a MIDI player
and decide to listen to John Cage “4:33” piece [1].
While you could assume the music was “playing” although it ended right
in the start, it easy to see that if it’s part of a playing list it’s
not the right thing to do.

Regards,
~Nuno Lucas

[1] http://en.wikipedia.org/wiki/4′33″On Sun, Aug 7, 2011 at 15:16, Mason Wheeler wrote:

From: David Olofson
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch
On Sunday 07 August 2011, at 15.00.45, Mason Wheeler wrote:

8

Just an example where this is just bad: you implement a MIDI player
and decide to listen to John Cage “4:33” piece [1].
While you could assume the music was “playing” although it ended right
in the start, it easy to see that if it’s part of a playing list it’s
not the right thing to do.

Regards,
~Nuno
Lucas

Given the way a MIDI file is structured, my patch would not actually
affect a hypothetical MIDI of 4:33 unless it was implemented in a very
strange way.? Basically, it watches for the first Note On and makes
sure it plays right at the start instead of after a delay.? 4:33 has no notes,
so it wouldn’t get trimmed.>From: Nuno Lucas

Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

9

Just an example where this is just bad: you implement a MIDI player
and decide to listen to John Cage “4:33” piece [1].
While you could assume the music was “playing” although it ended right
in the start, it easy to see that if it’s part of a playing list it’s
not the right thing to do.

Regards,
~Nuno Lucas

Given the way a MIDI file is structured, my patch would not actually
affect a hypothetical MIDI of 4:33 unless it was implemented in a very
strange way.? Basically, it watches for the first Note On and makes
sure it plays right at the start instead of after a delay.? 4:33 has no
notes,
so it wouldn’t get trimmed.

You are assuming things, which is not good for library writers (or any
programming in general).
Many MIDI files are computer generated, so can be very “strange” indeed.

If you can’t control the input then you must assume it can be anything.

Regards,
~Nuno LucasOn Sun, Aug 7, 2011 at 18:14, Mason Wheeler wrote:

From: Nuno Lucas <@Nuno_Lucas>
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

10

Before we jump to too many conclusions here, how about this idea:
Add a new playback flag for skipping silence at the start.

This leaves the decision up to the app, providing both approaches, chances are performance midi apps are quite different than typical uses of midi music on consumer computers.–
LordHavoc
Author of DarkPlaces Quake1 engine - http://icculus.org/twilight/darkplaces
Co-designer of Nexuiz - http://alientrap.org/nexuiz
"War does not prove who is right, it proves who is left." - Unknown
"Any sufficiently advanced technology is indistinguishable from a rigged demo." - James Klass
"A game is a series of interesting choices." - Sid Meier

11

Forest’s idea is not a bad one, but then it should also be implemented for all other kinds of music on all platforms that SDL_Mixer supports, for consistency’s sake.

It will probably less work just to edit the input files and get rid of that leading silence if you don’t want it, though, than to implement such a feature in SDL_Mixer…

Forest Hale wrote:> Before we jump to too many conclusions here, how about this idea:

Add a new playback flag for skipping silence at the start.

This leaves the decision up to the app, providing both approaches, chances are performance midi apps are quite different than typical uses of midi music on consumer computers.

12

Only if I have direct control over all music to be played.? Which I don’t, since I’m writing an engine, not the game itself.? So it has to accept whatever’s given to it and play it properly.________________________________
From: beoran@gmail.com (Beoran)
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

Forest’s idea is not a bad one, but then it should also be implemented for all other kinds of music on all platforms that SDL_Mixer supports, for consistency’s sake.

It will probably less work just to edit the input files and get rid of that leading silence if you don’t want it, though, than to implement such a feature in SDL_Mixer…

13

Hello !

I just submitted a patch at http://bugzilla.libsdl.org/show_bug.cgi?id=1267 that improves MIDI playback in two ways:

  1. It trims leading silence from the song. Some MIDIs begin with a few seconds of silence, which can make working with them in a game awkward.
  2. Fixed the playback code to respond properly to Controller Event #111, the looping command.

The patch only applies to native_midi_win32.c, but the logic changes are simple enough that they could probably be ported to any of the other playback engines.

-1 from me

I think this is a bad idea. Where does it stop ? If one part has such a general feature,
others should have too, like to get rid of silence in samples at the beginning. What about MODs ? XMs ?

In my eyes a better way would be to load the MIDI into a
memory buffer and patch it there, to get what you want.

Everbody that releases a game using his own code or an engine
of somebody else has to adjust his art- and soundwork, this is
one part of making a game.

The same thing, i would not like to have a function in SDL that crops a SDL_Surface to
its own content, like every Scanner Software does. This is something that should be done externally.

CU

14

Your best bet then, for total control (as far as that goes with unknown input)
is probably to use a specific software synth, such as FluidSynth.

If all else fails, you might fast forward the audio stream (ie running the
callbacks full speed, discarding the output) until audio is detected, and then
revert to normal operation. That way, the sequencer/synth won’t even be aware
that anything unusual happened.

But, quality sound consumes quite a bit of CPU power. You can easily have
"studio quality" on a proper PC these days, but Intel Atom or weaker might be
problematic…On Monday 08 August 2011, at 15.03.03, Mason Wheeler wrote:

Only if I have direct control over all music to be played. Which I don’t,
since I’m writing an engine, not the game itself. So it has to accept
whatever’s given to it and play it properly.

From: Beoran
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

Forest’s idea is not a bad one, but then it should also be implemented for
all other kinds of music on all platforms that SDL_Mixer supports, for
consistency’s sake.

It will probably less work just to edit the input files and get rid of that
leading silence if you don’t want it, though, than to implement such a
feature in SDL_Mixer…

–
//David Olofson - Consultant, Developer, Artist, Open Source Advocate

.— Games, examples, libraries, scripting, sound, music, graphics —.
| http://consulting.olofson.net http://olofsonarcade.com |
’---------------------------------------------------------------------’

15

Correct me if I am wrong, but it seems that this “silence at the
beginning” issue (1) came up when MIDI was looped after fix (2) provided
support for event #111. It might make sense to me implement (1) for the
2+ iterations of a MIDI loop if that is the case?

I would vote however for a manual implementation of such functionality
by adding a generic function such as
Mix_SkipSilence() - playback instantaneously skips over silent
parts in MIDI, WAV,…
which can be called manually by the game author if needed.

–AndreasOn 8/8/11 7:26 AM, Torsten Giebl wrote:

Hello !

I just submitted a patch at
http://bugzilla.libsdl.org/show_bug.cgi?id=1267 that improves MIDI
playback in two ways:

  1. It trims leading silence from the song. Some MIDIs begin with a
    few seconds of silence, which can make working with them in a game
    awkward.
  2. Fixed the playback code to respond properly to Controller Event
    #111, the looping command.

The patch only applies to native_midi_win32.c, but the logic changes
are simple enough that they could probably be ported to any of the
other playback engines.

-1 from me

I think this is a bad idea. Where does it stop ? If one part has such
a general feature,
others should have too, like to get rid of silence in samples at the
beginning. What about MODs ? XMs ?

In my eyes a better way would be to load the MIDI into a
memory buffer and patch it there, to get what you want.

Everbody that releases a game using his own code or an engine
of somebody else has to adjust his art- and soundwork, this is
one part of making a game.

The same thing, i would not like to have a function in SDL that crops
a SDL_Surface to
its own content, like every Scanner Software does. This is something
that should be done externally.

CU

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

16

Hello !

I would vote however for a manual implementation of such functionality by adding a generic function such as
Mix_SkipSilence() - playback instantaneously skips over silent parts in MIDI, WAV,…
which can be called manually by the game author if needed.

What about MODs, XMs ? They could have no notes at the start
or they could have notes there, but instruments with silence
at the beginning.

For other formats it is really hard to check
if there really is true silence at the beginning.

Even a wav can have real silence at the beginning,
but can have very low sample values, that are not zero.

CU

17

You’re wrong, since the 111 tag that shows where to loop back to is located at a point after the music has started playing.? They’re two separate issues; I just covered both of them in the same patch.________________________________
From: aschiffler@ferzkopp.net (Andreas Schiffler)
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

Correct me if I am wrong, but it seems that this “silence at the beginning” issue (1) came up when MIDI was looped after fix (2) provided support for event #111. It might make sense to me implement (1) for the 2+ iterations of a MIDI loop if that is the case?

18

Yeah, they could, but in practice, they don’t.? As someone pointed out earlier, this is usually done in MIDIs specifically to compensate for hardware limitations on certain really old MIDI hardware that’s not generally found in personal computers.________________________________
From: wizard@syntheticsw.com (Torsten Giebl)
Subject: Re: [SDL] SDL_Mixer: Improved MIDI playback patch

Hello !
?

I would vote however for a manual implementation of such functionality by adding a generic function such as
Mix_SkipSilence() - playback instantaneously skips over silent parts in MIDI, WAV,…
which can be called manually by the game author if needed.
? What about MODs, XMs ? They could have no notes at the start
or they could have notes there, but instruments with silence
at the beginning.
? For other formats it is really hard to check
if there really is true silence at the beginning.
? Even a wav can have real silence at the beginning,
but can have very low sample values, that are not zero.
? CU

SDL mailing list
SDL at lists.libsdl.org
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

19

That’s not the only reason. I’ve seen some music editors that export
to MIDI doing this when the first note is on an “incomplete” bar, so
they add the needed silence to complete the first bar.

My first guess about why, would be that this allows to simplify the
code when a metronome or chord harmony is added as a secondary track,
but I’m just speculating.

If I remember correctly, MIDI has no support for “incomplete” bars.
You could change the “Time Signature” for each bar, but that could
lead to strange things with computer generated harmonies, or
metronomes that take the “Time Signature” into account (e.g. to mark
the the first beat of a bar).

Regards,
~Nuno LucasOn Mon, Aug 8, 2011 at 16:20, Mason Wheeler wrote:

Yeah, they could, but in practice, they don’t.? As someone pointed out
earlier, this is usually done in MIDIs specifically to compensate for
hardware limitations on certain really old MIDI hardware that’s not
generally found in personal computers.

20

Speaking as a musician, all my midi files have leading silence. This is
because My recording studio uses outboard hardware that syncs up by a
clock in my hard-disk recorder, but the Midi sequencer is used to start
and stop all transports. So when I start to record a track, the
sequencer starts the external master clock and then listens, finally
starting a measure and a half later.

As for midi having no support for incomplete bars. It doesn’t need to.
Humans provide that support. It’s not unusual for me to record midi
tracks without any care as to their BPM or time signature. Consider
"Rhapsody in Blue". If you tried to create a timing track for a midi
sequencer to map all of it’s speed changes and measure lengths so that a
metronome could track them, you’d realize pretty quick that it’s much
easier to turn off the metronome and just let the musician create the
timing on the fly.

CWCOn 8/8/2011 10:39 AM, Nuno Lucas wrote:

On Mon, Aug 8, 2011 at 16:20, Mason Wheeler wrote:

Yeah, they could, but in practice, they don’t. As someone pointed out
earlier, this is usually done in MIDIs specifically to compensate for
hardware limitations on certain really old MIDI hardware that’s not
generally found in personal computers.
That’s not the only reason. I’ve seen some music editors that export
to MIDI doing this when the first note is on an “incomplete” bar, so
they add the needed silence to complete the first bar.

My first guess about why, would be that this allows to simplify the
code when a metronome or chord harmony is added as a secondary track,
but I’m just speculating.

If I remember correctly, MIDI has no support for “incomplete” bars.
You could change the “Time Signature” for each bar, but that could
lead to strange things with computer generated harmonies, or
metronomes that take the “Time Signature” into account (e.g. to mark
the the first beat of a bar).