I've gotten to the point where its time for non-inhouse testing of my latest game.

and for the first time in my life, i don't have any outside testers already lined up.

so i guess i'm gong to need to recruit some, or do a public beta, or something.

i mean, i need to get it into player's hands and get some feedback.

so.....

1. what should i do?

2. how should i go about it?

3. what should i watch out for?

4. in potentially releasing the game into non-trusted hands, what level of DRM protection should i include?

1. what should i do?
2. how should i go about it?
3. what should i watch out for?
4. in potentially releasing the game into non-trusted hands, what level of DRM protection should i include?

4. It sounds like you're planning to release it into the wild, rather than doing a normal focus group test (in which you set up some game systems at a marketing research facility, or do the focus group test in your own offices). That's more like Beta testing than focus group testing, and I can't advise you on copy protection schemes. That's a technical question for a technical forum.

3. You don't want to slant the respondents' answers by either (a) asking questions in a poorly worded way, or (b) having your game designer ask the questions and the testers know that he's the designer (people tend to be mindful of hurt feelings, and are likely to give him the answer he wants to hear).

2. A marketing research firm could handle this for you, but of course if you don't have a lot of money (or if you aren't going to make a lot of money with your game) that might not be an option for you.

1. Think of it as marketing research, read about how marketing research works, and ask very good questions that'll get you the information you really need. You want to know if people will buy your game and how much they'd be willing to pay (not "do you like it"). And you want at least 100 respondents, preferably more.

By the way, did you read the other thread on this forum, entitled "Focus Groups and Play Testing"?

That's more like Beta testing than focus group testing

yes, i was originally going to use the term beta testing, but was afraid of causing confusion, as there seem to be varying definitions of alpha and beta testing.

so here's where i'm at:

First, the game is a new version of a proven hit. The original version of the game was featured on one of the three major network's local evening news in the Nation's Capitol as a last minute Christmas gift suggestion (without my even knowing it!). By version 1.3 it had become so popular that it became a target for crackers. So no real need for focus group stuff. The game has been largely completed except for final improvements to graphics and audio, and implementation of a few high level things like optional quests and such. The game has already been tested and debugged in-house, as well as play tested. Bugs are very against my religion, so the list of "bugs" in the game consists of about a dozen minor imperfections in the current graphics that do not affect gameplay, and all of which i could probably fix in a single day. Usually at this point I have some people I know who are interested in trying out the game, and I run a sort of long term private beta(?) test program from now until release (in say 3-6 months). However this time, I don't have anyone lined up. and need to find testers. I'm not really looking for people to find bugs for me. As developer, that's my job before I can honestly declare the code to be done. I'm more looking for folks to try it out, and see what they think. Usually from these private betas all i get is "works great! cool game! when's the next update? whats going to be in it?". granted, not much, but it tells me i'm doing my job correctly, and there are no issues of any kind. basically, as long as i get that type of feedback (no bugs, fun to play, still playing it, want to continue to play it, looking forward to the upcoming features, etc). i know everything is going ok. Also, i do occasionally get some useful suggestions, which of course are invaluable.

So, now that i've hopefully made myself a little clearer...

1. what should i do?

2. how should i go about it?

3. what should i watch out for?

4. in potentially releasing the game into non-trusted hands, what level of DRM protection should i include?

Approximately how much do you intend to spend on this beta testing phase?

There's a number of avenues you could go, but it really depends on what budget you're running.

unfortunately, i have no testing budget. everything is being done on a shoe-string. a $400 baseline windows 7 box, and $50 a month internet - that's it. and electricity, of course.

limited version public beta demo posted online? hope for feedback and maybe get some interested testers that way?

It's definitely one viable option...

Is the game meant to be playable as a single player experience?

Is having people over to test specific interactions doable for you?

Can you afford to put some DRM and generate time-limited demo versions?

Also, while subjective feedback is very important, I'd also be concerned with actual metric-tracking implementation. It may or may not suit your particular game, but beyond what the player sees, or even an acute observer might uncover, there are a lot of things that make more sense once put in cheer numbers as statistics.

I'm going to assume (probably erroneously) that you're building an F2P game to explain this.

In an F2P game (for example), one of things that you really what to know is what's the most common process (series of operations) that a player undergoes before becoming a paying user, and how many players get "lost" during this process, where and why. Implementing metric tracking of any kind allows you to build funnels that help shed some light on that process. While you *could* go through the hours of play-test observation to uncover this, metrics could help you define it very quickly, and they remain even after the players have left your home. That's valuable, and especially if you have a bad memory like me

Is the game meant to be playable as a single player experience?

yes

Is having people over to test specific interactions doable for you?

not really.

Can you afford to put some DRM and generate time-limited demo versions?

sure, no problem. DRM was done FIRST - BEFORE the game was started! <g>.

In an F2P game (for example), one of things that you really what to know is what's the most common process (series of operations) that a player undergoes before becoming a paying user, and how many players get "lost" during this process, where and why.

yes, i once saw a posting from a causal user who was looking for CAVEMAN v1.x. He had tried the demo, but did about as well as any non-gamer would in a hard-core rpg. Yet here he was, 5 years later, posting online, looking to give it another go.

and yes, you assumed correctly, this is not an online f2p game. pc, single player. free limited demo - playable tutorial - can only play a character for 30 game days, can't load saved games.