Vaporwave is synonymous with music. It is primordial that music must be spot on... But with recent takedowns of Vaporwave classics, let's just say that I'm less eager to have sampled Vaporwave included: I don't want that to spoil an otherwise perfectly good game.

So I'm probably gonna ship the game with sample-free Vaporwave, made by yours truly.

However, that's not the end of it.

I'm going to let the user play their own Vaporwave if they like. That way I can secure myself from any takedown possible.

I'm also going a step further and created the 【 ＶａｐｏｒＭａｋｅｒ】

Unity and audio

As you're probably aware, Unity can play audio assets that are imported into the UnityEditor. This approach only works with packaged audio, however. This means that we need another way to play external files. 

There's a utility class that can be used to play any sound file from anywhere: WWW . This class is used to do simple HTTP requests and catch its results. 

If we use the "file://" protocol, we can actually load a file from the player's local machine. And what's more, is that there's a nice method for getting audio clips: WWW.GetAudioClip

Cool, let's use that.

WAIT, WHAT!?! MP3 ARE ONLY SUPPORTED ON PHONES!?!?

That's no good...

The workaround

So, you're telling me that mp3, the most universally available file format, is not compatible with Unity's audio system?

Yes. It appears so... Due to licences issues, Unity cannot be shipped with an MP3 decoder. Which is really weird, but we can't really do anything about it.

Thankfully, Unity has C# and .NET, which are one of the most used tools nowadays. I'm pretty sure that there exists a way to fix this.

Enter NAudio.

NAudio

NAudio is a .NET library that can load and play most audio files. This is really useful because we can then use that library rather than Unity's Audio system. 

NAudio is compatible with Unity, which is a big plus for us. Of course, we'll need to do a bit of fix around, but it's nothing really hard.

NAudio is just a .dll. It's just a matter of dropping it in our Asset repository and voilà: you can now use NAudio in our scripts.

Here's the little blog post I've followed if you're interested.

Reading metadata

Listening to mp3 is fun and all, but I also want the player to know which song is playing. Things like song title, artist, album title and even, if I can, the album cover...

Most of the time, these pieces of information already exists within the .mp3 file itself as metadata. This is how, for example, most media players are able to display the album cover of a song. Similarly, this is also how some applications are able to group songs of the same album together.

Depending on the used metadata convention, they are either at the very beginning of the file or at the end and can take many forms.

We won't need to open a byte stream and manually seek these metadata ourselves. There's already plenty of libraries that are able to do that for us.

Funny enough, the solution I've chosen came from the Linux world. Let me introduce you to Banshee.

Banshee is an open source media player not unlike iTunes. It can manage one's music collection and play them. This program is written in C#, which coincidently is the same language as our favourite engine...

The library responsible for reading such metadata in Banshee is called TagLib#.

With a little bit of tinkering, we can include TagLib# in our Asset repository, making good use of it.

After creating a UI element containing the metadata, here's the result:

(The blurriness is part of the art style, trust me)

The 【 ＶａｐｏｒＭａｋｅｒ】

Now we have both the data and the playback. But I'm not satisfied.

Managing the music

As you may (or may not know), Vaporwave is basically slowed downed music.

Some artist chooses to keep the editing at a minimum, while some add a lot of butter on top, but it all boils down to slowed downed music.

The idea I have is to let the player create their own Vaporwave to be used in the game by putting some mp3 file in a special folder. These files will be played at a slower speed than usual, thus creating some rudimentary Vaporwave.

Although the idea is simple, NAudio itself doesn't come with such functions... However, they DO have a post on the subject.

Basically, we'll add SoundTouch, an open source sound manipulation library. Although written in C++, we can still call its native function with the wrapper given by the post.

One drawback is that we'll need to supply native .dll libraries for all platforms if it's doable. A few extra resources (Like a mac and Linux installations) are needed for this if you want multi-platform support, but nothing really hard. (If all fails, you can just copy/paste the source code and fix things here and there)

So by following along the source code, we can add the required files in our Asset repository. Easy as pie.

Once everything is set up, we'll just need to plug that VarispeedSampleProvider class into our mWaveOutDevice instead of the mVolumeStream like so (if you follow along that blog post I've referenced earlier)

private void LoadAudioFromData(byte[] data, bool isVaporMaker)
{
	MemoryStream tmpStr = new MemoryStream(data);
	mMainOutputStream = new Mp3FileReader(tmpStr);
	mWaveOutDevice = new WaveOut();

	if (!isVaporMaker){
		mVolumeStream = new WaveChannel32(mMainOutputStream);
		mWaveOutDevice.Init(mVolumeStream);
	} else {
		mSpeedControl = new VarispeedSampleProvider(WaveExtensionMethods.ToSampleProvider(mMainOutputStream), 100, new SoundTouchProfile(false, false));
		mSpeedControl.PlaybackRate =  0.75f;
		mWaveOutDevice.Init(mSpeedControl);
	}

}

When we'll play the song, it'll play at any speed we specified by the PlaybackRate property of the VarispeedSampleProvider instance we constructed. 

Managing the art

I've could have stopped there, but I STILL wasn't satisfied. I wanted to make a clear distinction between normal custom music and any piece that went through the 【 ＶａｐｏｒＭａｋｅｒ】. 

To do so, I've decided to change the album cover for something more vaporwave.

When I fetch the album cover for vaporized songs, I actually map each pixel by their lightness to a gradient of two colours. I've also made those colours members of the MonoBehaviour so that they are available in the UnityEditor.

TagLib.File file = TagLib.File.Create(filepath);
TagLib.IPicture pic = file.Tag.Pictures[0];

Texture2D text = new Texture2D(2, 2);
text.LoadImage(pic.Data.Data);

for (int x = 0; x < text.width; ++x)
{
	for (int y = 0; y < text.height; ++y)
	{
		float h,s,l;
		ColorExt.RGBToHSL(text.GetPixel(x,y), out h, out s, out l );
		text.SetPixel(x, y, Color.Lerp(m_vaporMakerDarkCoverColor, m_vaporMakerLightCoverColor, l));
	}
}

text.Apply();

Afterwards, when the UI displays the album cover of the vaporized song, it will use the funky artwork rather than the original one,

Here, take a look:

     

 

With all of this, I'm sure that this feature will be popular, If not only for the ability to play our custom music.