I'm curious as how games like Audiosurf and Vib Ribbon generate their stages. I know Audiosurf's tutorial gives a basic description of what it takes into consideration. But for a designer, what exactly are the proper terms of the components taken into consideration? What exactly are the parts of a digital audio it looks for? Sorry if my question lacks detail, I don't know much about audio/music especially digital audio.

Better answers should be expected by posting in Music and Sound, methinks.

Yeah at first I was thinking of posting it there, but then I realized this is a game design related issue so I posted it here.

Quote:

Original post by qcfx2a Yeah at first I was thinking of posting it there, but then I realized this is a game design related issue so I posted it here.

But you asked a technical question about audio tools. Therefore it belongs here in the audio forum. Hope you get good answers here.

My guess would be that the waveform is converted into frequency domain, and then the "blocks" on the stage (I'm thinking of audiosurf) would be placed whenever there are spikes in the frequency domain above a certain threshold. Whenever a drum is hit or a note is played on an instrument it will be very visible in time-slices of the spectrum in frequency domain.

The mathematics are beyond what I can go into here, but the basic operation would be to apply "Discrete Short Time Fourier Transform" to the input sound waveform. This will convert a "time domain" representation of a song (amplitude vs time of the waveform) into the "frequency domain" (amplitude vs frequency for a short slice of time). You can do analysis of the frequency domain to get an idea of when notes are played in the music, how fast are the notes, are they high pitch or bass, etc.

Sorry if this is over your head, this can get to be some pretty heavy college math, but this is basically what you need to learn to analyze a digital audio file. Other things you might want to search for is "Fast Fourier Transform (commonly 'FFT')", or generating a spectrogram.

Hope this helps.

I've only played AudioSurf once, but as the poster above says i'd imagine it would use FFT as a starting point for analysing the music. The FFT is calculated every few thousand samples (which depending on resolution will represent a slice of the audio for just a few milliseconds). With snapshots of the frequency you can start to analyse what the music is doing, such as beat finding.

A good article on this, although the maths is somewhat complicated can be found here on gamedev

From here you can start to examine what the music is doing in say the bass, middle and treble ranges. You can take this further by using audio processing such as filtering, EQ etc to extract a specific range of frequencies, such as drums (bass), snare, hi hats etc. Next you'd need to consider the changes between snapshots, in order to build a picture of what is happening to the music over time, e.g finding beats.

This is where the 'art' comes into play as its the process you use and the decisions based on the results that construct the 'stage'.

Its actually quite a large field and if you want to gain better understanding it would probably be useful to start learning about digital singal processing, (DSP) etc.