Building the SlimDX Installer

Published June 19, 2008
Advertisement
The spy museum in DC is pretty cool, but it's not part of the Smithsonian. That is why:
1) It isn't on most of the Smithsonian maps, making the task of finding it somewhat irritating.
2) It charges $18 at the door.
I don't mind 1 that much, but 2 was really obnoxious.

Building the SlimDX Installer

Developing the SlimDX installer turned out to be a massive headache. I believe there is good installer creation technology out there; however, it would appear that I do not actually have access to it. When it comes to the world of install technology on Windows, there are basically two groups:
  • Installers that create Microsoft Installer (MSI) files. This includes the Setup Project in Visual Studio, and stuff created by InstallShield, Wise, WiX, and so on.
  • Installers that do their own thing. In this group are NullSoft, InnoSetup, and some others.
Now, the first headache is simply the task of selecting which installation system to use. I threw out InstallShield and Wise because they cost money. WiX was a huge pain and everybody seems to complain about it, so I didn't give it much time of my own. That left Visual Studio, NullSoft/NSIS, and InnoSetup.

The SlimDX installation had three basic requirements. First, it had to install the basic native prerequisites that SlimDX apps always require. For the June release, that meant the Visual C++ 2008 (version 9) runtime, and the DirectX redistribable files for June 2008. Second, it had to register SlimDX in the Global Assembly Cache (GAC). Lastly, it had to do an installation to a normal Program Files directory, which is necessary for VS to find the XML and PDB files, and for people to distribute SlimDX as a private assembly.

The first sticking point came when I was looking into registering SlimDX in the GAC. One would expect that this is fairly simple. Not so much. See, the usual method developers are shown for registering their stuff in the GAC is to use the gacutil.exe utility to do it. It turns out that this is only for developers. Normal users don't have gacutil, and you aren't allowed to redistribute it as part of the installer. So how do you register in the GAC? MSI can do it; if you're not MSI, you can go to hell. (There's a class in the .NET Framework that can do it, so if your installer can handle plugins written in .NET, then you can pull it off.) At this point, NSIS and InnoSetup dropped out of the running. No GAC registration was a non starter, and I didn't want to use a hack for something so simple and fundamental.

Okay, so now I was down to VS, which I didn't think was a bad thing. It generated these hideous, amateurish looking installers, but I found out that if you know where to look, you can reconfigure that stuff so that it actually looks like a decent bit of software. I mean it's VS, right? Stuff might be a little weird, but it's basically good at what it does? Not exactly. Visual Studio is a terrible installer authoring system. There doesn't seem to be any specific reason for it, either. It's just all around shoddy, and omits stupid things that should be simple. For example, there is an interface for working with special folders like the Windows directory without having to give them explicit paths. That's great...except there's no support for getting the temporary directory. There's no reason not to, because the MSI format supports it, and if you know the property name, you can write it explicitly in VS. You just can't get it automatically, so you have to check the documentation for what the properties are.

After tweaking the project for a while, I got back around to GAC registration. It turns out that Visual Studio doesn't look at the GAC when adding references. It actually reads a set of registry keys (HKLM or HKCU, SOFTWARE\Microsoft\.NETFramework\AssemblyFolders) and uses their default values to get a list of search directories, and then it loads assemblies that it finds out of those directories. So to install a managed assembly, you put it somewhere on the file system apart, then register it with the GAC and also add the assembly folders entry. VS even has a Registry view that allows you to do this sort of thing...except it doesn't let you edit the default value of a key. Again, this is something that is trivially supported by MSI, VS just doesn't do it. Unfortunately, there's no hack-around in VS itself. You have to install a tool called Orca to modify the MSI after VS finishes building it, and write in the registry value yourself. Orca does support automation, but I have yet to figure out exactly how.

Oh, and on Windows XP 64 or Vista 64, VS reads from HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432node\Microsoft\.NETFramework\AssemblyFolders instead, because it's a 32 bit app and it gets redirected. That took a little figuring out, because the installer runs as 64 bit and doesn't write under WOW6432node. What I haven't figured out, however, is why Windows/msiexec completely ignored me when I tried to write the key under WOW6432node. That's why 64 bit users need to browse to the SlimDX directory manually.

Installing the VC9 redist turns out to be rather easy, but the results are clunky. VS allows you to ask it to create a bootstrap EXE that installs requirements like the CRT and .NET -- but not DX for some reason -- automatically as necessary before launching your installer. All you have to do is check the boxes. Problem is, it doesn't package the MSI or the CRT redist into the EXE, like you'd want. It just leaves them hanging there, loose. I couldn't find any useful information on changing this behavior, or really doing much of anything about it. I ended up writing a simple EXE file that embeds the EXE, the MSI, and the VC9 redist folder inside itself. That's the published SlimDX installer; when you run it, it will stage those files to the temporary folder (typically Documents And Settings\USER\Local Settings\Temp), and invoke setup.exe there. The sad part is, despite being a total hack, this is probably far easier than more maintainable than any other way out there for packing things together, self extracting ZIPs included. (Although for this June release, the DX team finally got around to making their installer not a POS zip-inside-a-self-extracting-zip.)

There was also the question of installing the DirectX redistributable. There's a web installer...but that's not redistributable and so it's worthless to us. The normal redistributable package is about 70 MB, and includes every redist package since the D3DX library was made dynamic. That was in 2005. Somehow, I didn't really feel that SlimDX users needed DirectX from October 2005. It's just a waste of space and bandwidth. Luckily, the DX setup program is flexible, and allows you to reconfigure it and simply delete old components. I cut it down to a minimal set that doesn't include DX 9.0c core install files (which are huge), and that subset was fairly compact -- about 8 MB. Not bad. Now I just had to actually run that minima installation as part of the SlimDX installer, and everything would be good. It shouldn't be a surprise that there was precisely zero built in support for doing this.

After some research, it was clear that the only way to really do this was to write a "custom action" for the install. That doesn't sound so bad...except custom actions are either JScript/VBScript (highly unrecommended), or DLLs. (Managed DLLs can be used, and are handled via a native proxy DLL. Blech.) I decided to just follow this guide (which is poorly written, frankly) and build a C++ DLL called DXSetupAction to handle this for me. That way I'd even get to use the MS DirectSetup API, which can be used to easily and transparently install the DX redist. (Although the 1.5 MB dxsetup.exe file is required to be redistributed even if you do use DirectSetup. What the hell.) It was here that I realized that there was no clear way to access any of the DX redist files if they were embedded in the MSI. It sounded possible, but I couldn't really see how. (Later, I realized that the custom actions only run AFTER your application installation, so I could have simply installed the files as part of normal installation, even if that is rather contorted.) I decided to simply embed all of the redist files inside my DLL, just like I did with the main EXE, stage them to the temporary directory, and call DirectSetup from there.

I would love to say that worked, but it didn't. When testing on my machine, everything seemed fine. It was only when I moved to clean VM tests that I noticed that DirectSetup wasn't doing a goddamn thing. It would load out of the DLL. It would find the function. It would run the function, and it'd get a success code back. But it didn't actually accomplish anything. I decided to simply CreateProcess on dxsetup.exe with the /silent flag instead, which did work as expected. Maybe that's why you're required to redistribute the executable -- it saves you time once you notice their API is broken. On the bright side, I now have a single DLL which you can easily add to anything that can load it, MSI or otherwise, and it will quietly and neatly install DirectX for you. (The version's pretty easily configured, and it can do either x86 or x64.)

In any case, I finally had it working. Let's look at the layout: all of the DX files are embedded in DXSetupAction.dll. DXSetupAction.dll, and SlimDX.dll/.pdb/.xml are all embeddded in SlimDXSetup.msi. SlimDXRedist32.msi, setup.exe, and a vcredist_x86 folder are all embedded in SlimDXSetup.exe. I personally find this whole thing to be fairly ridiculous...but it works. This was a fairly simple install, too. I can't imagine what kind of horrors I'd have to endure if I was doing something more complex, like Visual Studio or Office. As I said before with the docs, I can't help but think that they must have something better than what the non-paying public has been given access to.
Previous Entry SlimDX Installer Teaser
Next Entry Ventspace
0 likes 13 comments

Comments

Tape_Worm
Yeah the Visual Studio install builder is ridiculously crippled. I tried using it to distribute Gorgon and boy was I sorry. Thus I switched to the nsis installer, not a great solution, but meh.

I thought you didn't want to put SlimDX in the GAC? Or at least I thought I saw you mention that at some point?
June 19, 2008 10:22 PM
jollyjeffers
Quote: The spy museum in DC is pretty cool, but it's not part of the Smithsonian. That is why:
1) It isn't on most of the Smithsonian maps, making the task of finding it somewhat irritating.
[lol] Surely making a spy museum easy to find wouldn't really be in the spirit of things?!


Regarding installers... your adventures in this space seem to echo mine (although I had to interface with and configure IIS instead of DirectX) and I'm increasingly inclined to put it in the same box as Windows Security - "sufficiently difficult and irritating that its no surprise people don't do it properly" [headshake]
June 20, 2008 03:17 AM
Promit
Quote: Original post by Tape_Worm
I thought you didn't want to put SlimDX in the GAC? Or at least I thought I saw you mention that at some point?
I never wanted to require GAC installation, and I still don't. There's still a ZIP archive of just binaries available for developers, if you want. (I uploaded it a day later though.) It's just that the redist installation, which installs to GAC, is the recommended and easy approach.

The reason for this was that for a good chunk of SlimDX's lifetime -- and this is not news to you -- releases were more symbolic than practical. You really had to follow SVN to make sure the library was all patched up and working properly. With the March release and especially June, things are much more stable and it's fairly sane to use the released version, rather than tracking SVN.

Hell, we don't even have any high priority work items on the core library right now.
June 20, 2008 07:56 AM
mg_mchenry
Installers are hell. They are an art and discipline separate from everything else you do. I've worked on teams that had a guy dedicated full-time for the life of the project just to building the installer.

I've gotten pretty good at using the VS installer now, but just like you've described, you're always working around it.

In my previous plans, I was going to ignore the EULA for the DX web installer and distribute that until I got sued.

That was when I considered releasing an app on managed DX. Thanks for all your hard work, Promit.


One note about the GAC installation and the nullsoft installer: As you pointed out, there is a class/namespace/API in the framework you can use to register libraries in the GAC. You could have made a simple command-line utility that runs at the end of the setup process to do the job, and still used NSIS. I've never used it, but it's hard to imagine it can be as difficult as the VS setup project.
June 20, 2008 01:50 PM
Tape_Worm
Are you certain that the gacutil isn't allowed to be redistributed? I have the ComponentOne enterprise suite for .NET 2.0 installed here at the office and when I set it up it installed a copy of gacutil.exe in its bin directory. I wonder how they got around that?
June 20, 2008 02:12 PM
Promit
They probably are simply redistributing it in violation of the EULA. It's not redistributable as a legal limitation, not a technical one.
June 21, 2008 09:23 AM
Black Knight
I think all this setup project mess is just PITA.
It should be simple as selecting the directory of you executable and it should find the dependencies or at least it the setup project should let you select which dependencies to install.It only lets you install some of them which have product.xml and package.xml inside the bootstrapper folder.
Here is how I created an installer today with lots of pain.
I created a setup project and added all the game data files and the executables.Then from the properties/prerequisites I selected Visual C++ Runtime Libraries(x86).There was no option for directX here unfortunately.
So I needed a way to install the DX runtime too first I tried adding it to the same setup project and running it as a custom action.This works but all the DX files are installed on the user machine too.
So I removed them from the setup project and put them in a folder called DirectX under the setup project release folder.
Now I need to externally call DXSetup when my setup.exe is started O_O I couldnt find a way to do that easily so I created another win32 application without a window.It just runs another setup.bat file located in the setup folder.
The bat file does this :
"DirectX/DXSETUP.exe/silent"
"Data/SETUP.exe"

It runs the DXSetup.exe in silent mode.
And then runs the actual setup.exe which is the setup project.

The DirectX folder contains the minimum required files for the DX runtime these are :
DXSetup.exe
dsetup32.dll
DSETUP.dll
dxdllreg_x86.cab
dxupdate.cab
Jun2008_d3dx9_38_x86.cab (my application uses June SDK)

The Data folder contains:
vcredist_x86 (folder which contains (vcredist_x86.exe))
setup.exe (this is the actual setup exe from the setup project which is run byt the bat file)
Dark Age Setup.msi (MSI package)

Anyway I didn't test it on a machine without D3DX but it seems to run on my machine.
Still I prefer zipping the game and hope the user has brains enough to install DX Runtimes :P
June 30, 2008 09:46 AM
Tape_Worm
Quote: Original post by Promit
They probably are simply redistributing it in violation of the EULA. It's not redistributable as a legal limitation, not a technical one.


That's what I meant. I know they're capable of doing whatever they want with it. I'm just surprised no one's called them on it.
June 30, 2008 05:30 PM
Talonius
To install assemblies into the GAC at work I use the CmdHelper utility produced by Stephan Brenner. It requires a custom step using InnoSetup, but because of it's flexibility in how you register assemblies it's fairly simple.

In my installer if the user selected GAC Installation the custom script is fired at the end of the install process. The script first uninstalls all assemblies that have been signed using the appropriate key (allowing for rapid development of the assemblies without eight hundred copies of minor versions) and then installs all of the assemblies in the destination directory. With CmdHelper this is two lines of shell code; CmdHelper itself is included as part of the installer, and extracted to the Windows temporary directory.

I'm surprised that the installers haven't made more progress in integrating managed code management into their feature suite.
August 15, 2008 01:17 AM
legalize
Oh my, had I known you were attempting this for SlimDX, I would have offered my help up front. I only stumbled upon this now. I'm in a good position to help because I'm an MVP for DirecX and Windows Installer. I would have used WiX myself. If you would like me to work with you on improving the installer, let me know (just drop me a private message on gdnet).
August 27, 2008 01:50 PM
MattWorden
When time comes to distribute an app developed with SlimDX, end users will want to first run the SlimDX installer, then the apps installer, right?

Now that you've gone through the pain to build the complete SlimDX installer, is it an easy thing to bootstrap the SlimDX installer as part of the app installer? And, if I understand what I'm reading -- the end-user will not need to first install DirectX9 because the SlimDX installer takes care of installing all of the needed pieces ... do I have that right?

Thanks for your work on this, Promit (as well as the rest of the SlimDX team)!

-Matt
October 14, 2008 09:23 PM
Promit
Quote: Original post by MattWorden
Now that you've gone through the pain to build the complete SlimDX installer, is it an easy thing to bootstrap the SlimDX installer as part of the app installer?
I wish. The SlimDX installer has no silent mode and it can't really be embedded smoothly. Part of my work for November is to try and make a better install system. Don't know how that will work out.
Quote: And, if I understand what I'm reading -- the end-user will not need to first install DirectX9 because the SlimDX installer takes care of installing all of the needed pieces ... do I have that right?
Right.

By the way legalize, I believe I owe you an explanation of why I didn't like your guide. I'll put that together soonish.
October 15, 2008 04:45 PM
MattWorden
Thanks, Promit.

-Matt
October 15, 2008 08:48 PM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Advertisement