Then you’re gonna love me

Ooof, I’ve been busy. Family was in town, finals are looming, and I had to finish up my PhD proposal. I have two new music related posts to get up.

The first is my audio effect. I had to implement an audio effect from scratch. I decided to try making something that sounds like the batman voice, from Batman Begins and The Dark Knight. I ended up implementing a real time version of an effect called a Whisperizer. This was fairly non-trivial, as it required a Fourier Transform to create, and the FT mathematically requires a lot of time samples, which makes real time very difficult. I got it to work using a short time fourier transform (STFT) with some windowing and overlapping, and it sounds pretty decent.

I originally wanted to run it on a sample of Adam West, saying that line about on coming thrust of manhood from the campy 60s show, but I couldn’t find a sound clip. So here is me saying it:

And here is that same sample after I ran my effect over it. This actually works in real time, although it’s hard to show that with a browser post.

My next post, which is probably not gonna be today, is going to demonstrate the synthesizer i made.

Filed under: Music

Streeeeeeeeeeeeeetch

It’s pretty easy to stretch out a word by repeating a letter in the middle ad infinitum. Letters make for wonderful discrete units that are usually clearly and cleanly separated. It’s a little harder to do with a musical note.

Here is my unaltered note:

And here is one if I just grabbed a couple of random points in the middle, repeated the segment in between them a few times to make the note longer:

Obviously, that is totally useless as far as stretching the note’s length.

When we talk about a musical note, and the characteristics that define it, we usually refer to pitch. Obviously, there are other considerations, which is why a clarinet doesn’t sound like guitar. I’m not going to go in to a whole discussion of harmonics and psychoacoustics, but generally speaking the primary pitch we hear on any given note is the fundamental frequency. That is to say, the lowest frequency contained in the spectrum of that note.

In order to then stretch the note, if we can take a sample that is exactly one wavelength, we should be able to repeat the note. So first, we need to find the fundamental frequency, and then find a couple of points in the signal that are that distance apart. Guess what my assignment was last week.

Although this does work, sort of, it sounds somewhat artificial. I found that taking multiple periods was more effective. Too many periods, though, and the amplitude envelope of the signal will vary. God help you if someone is playing with Tremolo.

In addition to that, you get artifacts at the seems between the splices. Usually little clicks, but it’s hardly a consistent phenomena. I ended up building a cross fader, and overlaying each repetition with a window. This helped a lot and produced the following result:

My code was pretty weak when it came to someone playing with vibrato and tremolo. Here’s the best I could do with a trumpet sample which has both. The pitch is preserved, but the timber most definitely isn’t:

Filed under: Music

More of what I do

As promised, here is the podcast I made for Digital Audio Effects course. It’s a little less than 30 minutes, so should make for a nice diversion on a morning commute. I think it came out fairly well.

Audio Effects Podcast

Filed under: Music, Music

So what am I doing here?

I’m in London doing a Masters degree in Digital Music Processing.  Well, what does that mean?  In the simplest description, I use math to modify sound.  At a slightly higher level, I am learning all the math and algorithms necessary to perform audio manipulations that would be useful for any number applications, from designing an ipod to mixing a recording.

I recently had an assignment, that is relevant and maybe a bit neat to those who are interested in what I am doing.  I had to implement a time-stretch algorithm.  This is to say that I was implementing a way to slow down the tempo of a piece of audio.  The tricky part is doing so that the pitch doesn’t change.  Usually, when you slow down a sound file, the frequencies decrease and it sounds like Tardy The Turtle.

So here are three sound clips to show what I did.  My original code was just on raw .wav files. They were kind of big for the blog, so I’ve compressed them. I tried using aac but flash plugin bitched, so they are unfortunately mp3. The first is the original unaltered clip:

Then secondly, here is that same clip, but time-stretched by 20%.  This algorithm is designed to stretch the sound while maintaining the pitches from the original.  The downside is that high frequency transients of short duration, like a highhat or a snaredrum, get stretched and sound completely different.  Place close attention to the drum sticks at the beginning of the sound sample.

And here was the meat of my assignment.  I tried improving the sound quality, particularly of the percussive instruments, by a) detecting these high frequency transients and then b) not-stretching them.  The rhythm is going to sound a little wonky, but I still think it’s a significant improvement.

I have carried on further, and tried stretching the sections after the onsets an equivalent amount to improve the rhythm.  While it does make the rhythm sound correct, I’ve now introduced little gaps in to the playback, and I don’t think it sounds as good.  I’m sure I could correct this by spreading around the over stretching, but it was more work than I really wanted to put in to this assignment.  Maybe on the next one.

My assignment this weekend is to make a podcast, so you can all expect to see that show up on here eventually as well.

Filed under: Music