Advertisement

Questions for a tester.

Started by March 11, 2013 07:38 AM
8 comments, last by Tom Sloper 7 years, 7 months ago

I am doing a beta test for the game I am currently working on with friends and family as well as various people within my game dev community. I wanted to get impressions from them on how they felt about the game. I'm already getting metric data within the game but I want their opinions as well. I am using a Google Form for the survey so I have access to web inputs such as radio buttons and check boxes.

So I'm wondering what would be some great questions to ask?

Edit::

So I started writing some down, here's what I have so far.

1.How do the controls feel?

2. Are the controls easy/intuitive? (1 - 5 scale)

3.Why did you pick the character you played as?

4.How difficult was Level (1, 2, 3)?(1- 5 scale)

5.How fun was Level (1, 2, 3)?(1 - 5 scale)

6.What did you enjoy about the game?

7.What did you not enjoy about the game?

Hi!

First, some clarity would be nice.

What kind of game is it? So we can come up with more well rounded questions for you.

Did you develop the game with family and friends or are family and friends helping test the game?

If family and friends are testing, they will lie to you about your game. They'll do it to soften the blow of things they don't like or over-look little mistakes.

They have the best intentions but it'll taint the data.

Otherwise, I would tell the tester that their data will be anonymous and that most data will be viewed by an outside analyst. This helps create a larger disconnect from the game and the player so that they feel safer giving honest questions. Another good idea for testing would be to watch people play the game through a web-cam or some other non-intrusive way.

This will help you to see their reactions to certain aspects of the game that they might forget by the time they fill out their survey. Ideally, you'd want to have testers in person so you can ask questions of them when they're done rather than an online process, but I know that isn't always possible.

Oh, and ask questions about the over-all feel of the games pace and progression. This will give you information on the difficulty, level design, and even on whether or not certain mechanics are being utilized well. This kind of question is best asked as a short paragraph response answer.

Check out my game blog - Dave's Game Blog

Advertisement

So the game is a platformer for mobile that is using a non virtual joypad control input and it will be coming out for both iOS and Android. It's

has the general theming of a Mario game, so not dark like a Castlevania title.

The only other person that is working on the title is my girlfriend and I believe most of the people that are coming are going to be game

devs from my local IGDA chapter. We're also doing this at a local café so we may not be able to get webcam stuff going.

Also how many questions might be too many? We have them going through three levels that are approximately a minute and a half each

so I don't want to overload them.

Also how many questions might be too many? We have them going through three levels that are approximately a minute and a half each

so I don't want to overload them.

If you are open to bribing your testers you could present 3 separate questionnaire lists and say the basic one gets you nothing -- the medium length questionnaire gets you a coffee -- and the uber complex one gets you cake (as well the coffee).

Also how many questions might be too many?

you might be better served actually googling questionnaires - for structure, size, style etc there is quite a lot of information on this topic out there given how often they are used.

However with regard your Beta Test

From a simplified point of view a Beta test is about trying to identify bugs for removal that still remain in the game before release. Asking for value judgments about such things like "How difficult was level 1" are questions you would use in a usability test. The point is if the non-bug, value-judgement feedback you receive from your testers will result in you changing features in the game then you haven't truly reached Beta stage.

So the question is - are you truly at the Beta stage considering the questions you have mentioned up top? If you aren't then by all means suck every drop of feedback you can obtain.

So the question is - are you truly at the Beta stage considering the questions you have mentioned up top? If you aren't then by all means suck every drop of feedback you can obtain.


I guess it's slightly both. We're preparing a demo to show off but I also want to get players impressions of how the game generally feels.

So perhaps alpha would be a better term for this.

Any tips on what to google search? I have listened to some GDC talks before on the subject but never have found any questionnaires specifically for games

with the exception of club nintendo.

http://pocketcyclone.com/2010/08/03/surveys-for-beta-testers-i/

http://www.docstoc.com/docs/80007606/Beta-Test-Questionnaire-(DOC)

http://www.gamestudies.org/0501/davis_steury_pagulayan/

https://wiki.ucar.edu/display/dd/Player+Feedback+Questionnaire

You could always try this trial out

you are right in that there is little around - However my earlier point was more pointed towards finding how to build an effective survey as a general lesson applicable.

Advertisement

Instead of posting it as a poll, I would recommend giving open ended questions. While some poll data is valuable, as it focuses on what you want, ideally, your beta testers are there to help point out issues or potential that you didn't think of. Limiting it to a poll means you'll only get back the same ideas your thinking about already.

When asking a poll, make sure your asking questions that you can do something with the answers.

How do the controls feel?

I would expect answers like "good", "bad" and "confusing". If I received answers like "bad", that would tell me nothing about how to change it. You really need to ask why as well. Phrasing the questions differently can get people thinking more open endedly. Such as "Tell us about what you liked or didn't like about the controls."

Are the controls easy/intuitive?

This is another question where recieving the answers of "1" (not easy) only tells me that it needs to change, but not how. Essentially polls should be questions that if you recieve an answer, should tell you what to do next.

Here is an example of a great poll question. "Between Version [current] and version [previous], do you feel the controls have improved, gotten worse, or are about the same?" If I receive Improved, that tells me that the changes I've made are good, and I should keep them in. If I recieve Gotten Worse, then that tells me I'm headed in the wrong direction.

4.How difficult was Level (1, 2, 3)?(1- 5 scale)
5.How fun was Level (1, 2, 3)?(1 - 5 scale)

These are great questions. but let the individuals just answer for the levels they want to. I.e. if no one is saying anything good or bad about level 2, no reason to focus on it, when people are telling you that level 4 is too complicated. But diffuculty and fun are important measurements, that tell your designers they need to rethink this. I would recommend also asking what part of level X was particularly good/bad, and why?

*********************************

The second thing I would say is don't manage it yourself as a forum. Let the testers tell you exactly how they feel, and let them vote on the importance of other peoples issues. UserVoice.com is a free service for this. Users sign up, (you can also limit registrations to just approved beta testers)

But people will post a bug, a feature request or idea, and use a point to vote on it. people only get a few points. Then all you need to do is look at the items people vote as the largest issues. I've used this in the past with great success.

Moltar - "Do you even know how to use that?"

Space Ghost - “Moltar, I have a giant brain that is able to reduce any complex machine into a simple yes or no answer."

Dan - "Best Description of AI ever."

One other thing, is that your poll questions should change with what you were focusing on for that build/release. I.e. if you spent most of the time working on AI for shop keepers, your poll questions should be focused around that as well.

Moltar - "Do you even know how to use that?"

Space Ghost - “Moltar, I have a giant brain that is able to reduce any complex machine into a simple yes or no answer."

Dan - "Best Description of AI ever."

Not mentioned on any of those, there is enormous value in actually watching someone play.

  • Observe the buttons they hit quickly, and notice the buttons they hunt for.
  • Observe the text that they have to read slowly, and the text they ignore.
  • Observe the commands they issue more frequently or differently than you do.
  • Observe commands they never issue.
  • Observe when they get confused or frustrated.
  • Observe when they are excited or animated.
  • Observe when they smile and laugh, or frown, or scowl.
  • Observe when they discover a defect.
  • Observe when they want to do something but cannot.

You won't get those from surveys. Watching people at a playtest provides an amazing amount of feedback even if they never tell you a word.

This topic is closed to new replies.

Advertisement