the game life cycle

Key Chapter Questions

• What is the definition of a bug?
• What are the primary severity levels associated with bugs?
• What are the phases in the standard life cycle of a game?
• What is the distinction between closed and open beta?
• What are the distinctions between different testing disciplines?

Game testing might seem simple and self-contained, but the truth is that testing is akin to engineering and programming. No wonder the official title for game testers is sometimes "Jr. Engineer"! Some of you will actually need programming skills to succeed at testing. Others will need an eye for detail--or the succinct prose of a journalist. In this chapter, we'll uncover the world of bugs. We'll also tackle definitions and severity levels, and examine how testing relates to the life cycle of a game--while looking at various testing methodologies.

Bugs Defined

Let's hear it for the dictionary definition! According to Merriam-Webster, a bug can be many things:

1. a) an insect or other creeping or crawling invertebrate (as a spider or centipede)
   b) any of several insects (as the bedbug or cockroach) commonly considered obnoxious
   c) any of an order (Hemiptera and especially its suborder Heteroptera) of insects that have sucking mouthparts, forewings thickened at the base, and incomplete metamorphosis and are often economic pests--called also true bug
2. an unexpected defect, fault, flaw, or imperfection (the software was full of bugs)
3. a) a germ or microorganism especially when causing disease
   b) an unspecified or nonspecific sickness usually presumed due to a bug
4. a sudden enthusiasm
5. enthusiast (a camera bug)
6. a prominent person
7. a crazy person
8. a concealed listening device
9. a weight allowance given to apprentice jockeys

Definition #2 is the winner here; since a video game is a form of software, it fits like a glove. Let's take a closer look at this definition:

• unexpected: No one creates bugs on purpose; if it seems like it was done on purpose, it's an Easter egg--a different animal altogether.
• defect, fault, flaw, or imperfection: Something is wrong; it makes the game worse, ugly, or unplayable.

In games, every element that does not enhance the game but detracts from it is a bug. Sometimes a bug is almost im-possible to see--and sometimes it's right in your face, taking up the entire screen. Or maybe it's an audio bug and you'll hear it if the 5.1 surround sound is at full volume, shaking the whole building. When you're a game tester, it's your job to locate all the bugs you can, replicate them--make them occur again, at will--and write a report for the developers. You are the eyes and ears for the game they are making.

Your main purpose as a tester is to:

• find bugs
• replicate them
• report them

Secondary objectives are to:

• verify that the bugs have been fixed
• make sure the game is fun

If you keep these directives in mind, you'll become an extraordinary tester. Now it's time to take the first step into the "inner circle" of testing. Let's start by taking a closer look at bug severity levels.

Edward Rotberg on the Tester Who Saved the Day

Ed Rotberg has been in the game business since 1978, working as a producer, designer, programmer, and executive manager at various times in his career. He was one of the founders of Videa, which was eventually acquired by Bally Midway. His career has also included a two-year stint at Apple Computer. He has worked at Atari (Atari Baseball, BattleZone, Star Wars, Blasteroids, S.T.U.N. Runner, Shuuz, Steel Talons, Guardians of the Hood), Bally Midway (Snake Pit), 3DO (Station Invasion, IMSA Racing [M2]), Silicon Entertainment (NASCAR Silicon Motor Speedway), THQ (MX Superfly, WWE Crush-Hour), and Mine Shaft Entertainment (High Heat Baseball 2002, Nokia Pro Series Golf, Blazing Angels: Squadrons of WWII).

While I was at 3DO, we were doing an "edutainment" title on a rush schedule. There was a lot of video content to the game, which was targeted toward grade school children. In one of the video sections that was parodying a TV talk show, the audience (not the player) was urged to call an 800 phone number that spelled out a cute and appropriate phrase. Fortunately, on his own initiative, one of our testers decided to call this number. It turned out to be an adult hot line, and we had just enough time to change that video content before the game shipped.

Bug Severity Levels

To get things going, let's first learn about the four different severity levels associated with bugs. The nomenclature may differ from developer to developer, but respective meanings are virtually identical.

Low Priority Bugs

Low priority bugs (or low bugs) hardly matter; in fact, it often makes no difference to the development team whether they are fixed or not. Examples of low bugs are minor graphical glitches, a short sound distortion here and there, some small error in the user interface (including a typo, which would personally frustrate us!)--any "error" that does not impact the flow or the way the game is perceived in a major way. Low bugs abound in every single game you've played, from low-budget to AAA; they are sometimes ignored in favor of more serious bugs--the ones that do make a difference.

Medium Priority Bugs

Medium priority bugs (or medium bugs) should be fixed. They occur more often than low bugs--and they can almost always be counted on to happen. So if a visual bug happens often, instead of rarely, it might be upgraded to medium. At the same time, a bug can be categorized as low if it occurs deep in the user interface; the same bug might be a medium bug if it occurs on the title screen. Another example is a bug that would annoy a player, but not affect gameplay.

High Priority Bugs

High priority bugs (or high bugs) must be fixed. A game that ships with high bugs is either a bad game or it has been rushed to market. High bugs seriously affect gameplay; they make everything stop in its tracks. If you can't open a door and that same door ends the level, you got yourself a high bug. If you can't change weapons, that's a high bug. If the frame rate drops to 20 fps instead of the usual 60, guess what? It's a high bug.

Bugs That Won't Die

The longevity of some bugs amazes me. I've had players of MUD2 complain about some strange but obscure goings on which, when I finally tracked them down, were in major pieces of code that have been executed tens of thousands of times a second for 15 years. It's astounding that the program ever worked at all, let alone that it managed to do so robustly for so long. This isn't an isolated incident, either; it's happened at least half a dozen times. I look at the code and think, "Why didn't it instantly crash?!"--but this somehow managed to be avoided.

Dr. Richard Allen Bartle (Visiting Professor in Computer Game Design, University of Essex)

Critical Bugs

Critical bugs are very special. A critical bug demands immediate attention from the developers. Nobody goes home--and if they're already home, they better come running back. Critical bugs are responsible for "the sky is falling" 2:00am phone calls and other general chaos! But what is a critical bug, really? Critical bugs cause "disasters" such as crashes, freezes, and data corruption. If you ever had a 50-hour saved game corrupted in one sweep, you know what we mean. Critical bugs must be fixed at all costs--and if the publisher spots a critical bug during compliance testing (discussed later in this chapter), the game is sent back to the developer.

Testing at Every Stage

Testing is a critical component to the process. In fact, you should test at every stage of development. At NCsoft, QA worked directly inside our Scrum process teams (which meet frequently and quickly to address problems and update everyone on progress) and tested content as it was being created before we even sent the game on to publishing QA.

Starr Long (Executive Producer, The Walt Disney Company)

Testing & the Game Life Cycle

Games are living, breathing pieces of software. They all begin with a seminal idea that evolves into a prototype. If well-fed, this prototype grows and grows--soon becoming a pre-production game. As the game matures inside the developer's belly, it's eventually ready to go out in the world--usually with a big "push" from the marketing and PR departments.

For further reading on this topic, please see Game Project Management (Hight/Novak)--part of the Game Development Essentials series.

Concept / Design

In the concept phase, the designer, producer, or other "vision" person heading the project has an idea for a game. A short concept document is written that describes the idea to other developers. Concept art is often also supplied that focuses on a few characters, interior/exterior environments, props, and structures. If proof of concept is successful, the concept document will usually evolve into the game design document (GDD). The development of this document oc-curs during the design phase of a project.

Production / Prototype

Once an internal production green light is given, the developer works on a prototype of the game. The prototype is very rough and not really a game at all, but it contains a sample of the gameplay. If a publishing deal is not yet in place, the prototype will be shopped around to publishers and prospective investors--a process that sometimes condemns a prospective game to years in "limbo." It's essential that the prototype contains the gold in the game--that shining gameplay mechanic/story hook that can make a game financially successful.

Alpha

The alpha phase comprises a game's first major milestone, which is usually written into the contract (i.e., developers don't get paid if certain requisites are not met). The alpha phase demands feature completion, which includes a full playthrough (play a game from beginning to end) but often includes placeholder and temporary assets. Contrary to what some may think, the alpha phase is more about locking in the features than having a bug-free game.

We are in the process of cross-training our QA people to also do network monitoring and administration. This will prepare them to run more "live" games such as Webkinz that we expect to become more commonplace.

Jason Kay (Chief Operating Officer, RKG Games)

Beta

The beta phase is perhaps the most important milestone; it is roughly the time when all assets are in place and the game is virtually finished. A game doesn't fully hit the beta phase until all assets are permanent and all high bugs have been addressed; therefore, the days before beta can be terrifying--especially if you take into consideration that a big chunk of the developer's funding is conditional on hitting the beta phase on time. After hitting beta, developers are able to embark on something the industry calls polishing.

Testing is essential to presenting the game or milestone to the publisher for review. If we cannot get this right, then what kind of product will we be giving our customers who spend their hard-earned money to buy our game?

David Dawson (Environmental Artist, Snowblind Studios)

Gold

When a game hits the gold phase, it's ready for market. Bear in mind, we didn't say "done"--but it still can be shipped. Today, with consoles perennially connected to the Internet, games are regularly released in "almost ready" state and then fixed up though downloadable updates (or patches). Unfortunately, this doesn't result in better games. (Remember Test Drive Unlimited--which was unfinished when it hit the streets and would only be really finished about six months after launch?) Even so, gold is still seen as the time when a game has fully matured.

Time Management During the Development Cycle

Invariably the most challenging testing-related experience is proper time management as it relates to the development cycle. Project managers and producers need to ensure that milestones are adhered to and that the QA schedule not be viewed as a buffer that can be easily squeezed at the end of the process. Far too often, we see this attitude permeate developers--and ultimately the most successful products that launch are those that are given adequate resources and are simply managed well. Executive management will frequently make decisions that are not in the best interest of the consumer. While it must be recognized that the bottom line needs to be considered, a product's shelf life and longevity ultimately comes down to word-of-mouth and reviews. Companies such as id Software and Blizzard recognize that quality comes first and they are not beholden to deadlines; very few companies, however, have this luxury. But there is something to be said about that mindset and the importance they place on quality product.

Jerome Strach (QA Manager, Sony Computer Entertainment America)

Beta Testing in Detail

Besides being a major milestone, beta testing can be divided into two varieties: closed and open. Closed betas are used to protect a game from the general public, while still involving a very high number of testers. If a closed beta is suc-cessful, a developer might opt to move into an open beta.

Closed Beta

Closed beta (sometimes known as internal beta) means that no one outside the developer and publisher has access to the game. Most games go though beta quietly, rushing to make the release date. Beta testing, in this case, is very pri-vate.

Some developers spend time polishing the game during closed beta. Now that the game is feature complete and feature locked, developers can ensure that all aspects of the game work as intended--and that gameplay is much more fun. Requiem: Bloodymare is a title that did exactly that: Gravity used closed beta to fine-tune the gameplay and get rid of networking bugs, preparing the game for a successful launch. Even the best features can mean nothing if the implementation is faulty.

Open Beta

Open beta (also known as public or external beta) is often utilized for the rising number of games that have online components. Some genres such as MMOGs (or MMOs; massively multiplayer online games) are built on the online component. Connectivity and network issues can be paramount to an MMO's success. With the game growing expo-nentially, production testers and QA are simply not able to test the whole game--even if you have 1,000 testers on hand. External help is needed--and these external beta testers take part in open beta. External testers are not paid for their work; playing an early copy of the game for free is seen as enough of a reward. Open beta is used to stress test servers (push the servers to the limit with an extreme number of users), balance gameplay, find low and high bugs, and make sure the game runs properly when thousands of players are trying to "kill and maim" at the same time. Some closed betas are highly sought after (e.g., Age of Conan), while others have trouble gathering enough players--with adverse consequences once the game hits stores.

James Owen Lowe on QA Integration

James Lowe is a content designer and writer for Icarus Studios working on the post-apocalyptic MMO Fallen Earth. He has a BA in Communication & Culture from Indiana University and a BS in Game Art & Design from the Art Institute of Pittsburgh-Online Division (formerly the Art Institute Online). He lives in the Triangle area of North Carolina with his wife Mandy and daughter Nora.

From the point of view of a writer and content designer, I feel that one of the best things we've been able to do in the development of Fallen Earth is to integrate multi-disciplinary teams, including QA, into the content development process. Our content development teams are primarily composed of writers and content designers, world builders, scripters, and QA. As a new major content unit is being built, such as a town or level, QA is able to check the world area for art-related issues (such as collision, bad textures, etc.) while the writing and scripting teams prepare missions and other interactive content. As the events and missions are being implemented, QA is right behind--testing missions, finding oversights, and providing near-immediate feedback. This allows for faster iterations in the content development--providing greater, more focused polish over a shorter period of time than if we turned over a com-pleted content unit for QA to test. As content development teams move on to other tasks, the QA team keeps going back over major content units, along with all other systems--continually providing feedback and discovering bugs, doing team and multiplayer testing, and assessing the whole "fun factor."

Jerome Strach on QA Collaboration

As an active gamer for over 30 years and a working professional in the game industry for over 20 years, Jerome Strach has witnessed much growth and technological advancements. He earned money from his paper route to buy his first Atari 400 in 1980, and he recently was honored to participate in the final QA approval for Metal Gear Solid 4 and Grand Theft Auto IV. Working for companies such as Atari, Digital Pictures, Pixar Animation Studios, and Sony, Jerome was given a chance to explore many facets of game development--everything from quality assurance to programming to producing, which included working alongside some amazingly talented individuals. Believing life is a continual education process, he is now committed to delving deeper into information technology and network administration to hone his skills as the industry increasingly utilizes online connectivity and incorporates both social networking and casual interaction into games.

First-party development (companies such as Sony, Microsoft, and Nintendo--which are hardware manufacturers, publishers, and developers) has a large quality assurance (QA) department--and as with any company, that depart-ment is critical for providing valuable feedback. Fortunately, Sony values QA--and this support originates from Ja-pan. It should be noted that QA departments cannot inject quality into bad designs or poor concepts. Quality titles that are fun to play remain the sole responsibility of the game designers and producers to work out prior to getting far long the road of development. Focus groups (assembled by the publisher's marketing department to elicit feedback from a game's target audience provide feedback on a game's level of difficulty and fun, among other elements) should always be utilized early and often. Equally important is the need for developers to remember that the QA team is your ally and not your enemy. The interaction between the groups should be supportive and not combative. This is easier said than done when people are functioning on little sleep and living off energy drinks--but it needs to be said regardless. Collaboration will always make for a better product--but at the very least, the proper utilization of the skills and talent provided within the QA department can tremendously improve the success rate when looming manufacturing deadlines near and street dates haunt your dreams.

Production Testing & QA

On the surface, "QA" sounds like the basic quality assurance process that every product undergoes--whether it's HDTVs or toothbrushes. Before a product hits the market, it's essential that certain quality markers are met, if not surpassed. QA assures that. In games, QA works as a double- or even triple-check process, where a game goes though a very fine-tooth comb. Certain areas of the game are verified by more than one tester, and a rotation prevents anyone from getting used to a level and thus numb to possible bugs. QA is known a game's last line of defense.

Production testing is far from the last line of testing defense. Infantry is more like it! As a production tester, you will never get stable code. The game might not boot at all. Production testing is the process of making a build stable enough for QA and make sure that the game is fun to play. Therefore, a production tester must balance these two somewhat opposed objectives daily--being aware of technical issues and gameplay simultaneously. The bugs you will find as a production tester will be much more complicated than the bugs found by a QA team--that is, if you're doing your job right. The QA team is not supposed to stumble on bugs that should be found by production testers, in part because the sheer volume of a QA team (which can be upwards of 900 testers) makes QA expensive! A broken build results in money down the drain and happy testers playing ping-pong.

Testers & the Development Pipeline

Testers not only help us with the general problem of "removing bugs from the game," but they also help us prevent problems in the development pipeline--preventing developers from getting stuck with buggy builds of tools (or the game on their systems, unable to make any progress).

Brian Reynolds (Founder & Creative Director, Big Huge Games)

The QA-Testing Partnership

QA and testing go hand in hand with the larger game development process. You cannot have one without the other. QA is very important in getting out a game or expansion that is fun, bug free, and challenging all at the same time. The QA team does not create the game, but it can determine if the feel or vibe of a particular system is as intended from the dev side--as well as detect and report the bugs found. Others understand this and are more than willing (eager in fact) to cooperate with us to make sure their work is the best that it can be.

Floyd Billings (Assistant QA Lead, Sony Online Entertainment)

Testing Disciplines

As you learned in Chapter 1, testing started as a couple of people making sure the game worked. From there, it evolved into an essential part of a game's lifecycle--the "be all, end all" of a solid title. If your game had shoddy QA, sales would soon nosedive--and retailers wouldn't be happy. The testing disciplines listed in this section are areas of knowledge within testing--in contrast to testing techniques (which will be discussed in Chapter 6).

Balance Testing

Balance testing ensures that gameplay is fair to both the human and AI player alike. If one is too powerful, there's something wrong. In single-player games, balancing involves making sure that the "easy" level is not too easy, "hard" is not too hard--and "medium" offers a nice, gradual rise in difficulty. Multiplayer games need to be balanced as well. Maps need to be neutral in their design, weapons must be equal in power (if not, the characters wielding them should be), and spawn points need to be placed fairly. The truth is that without balance, all matches are skewed and no victory is fair--or fun. This is so important that developers sometimes spend months balancing the gameplay!

Balance testing demands acute attention to detail. At the end of the day, it's the feel of the game (and your notes) that will help the developer fine-tune the gameplay. It's a fun job--exhilarating sometimes, but a very serious task nonetheless.

How to Balance a Game

Here are some guidelines for balancing a game during the testing process:

• Make sure the code is stable. Crashes and freezes will make balancing impossible.
• Enlist an equal number of testers for each side. Make sure their skill levels are matched by members of the opposite team.
• To balance weapons, load a neutral map. To balance a map, have everyone equip weapons of the same class.
• Play for at least one hour, making notes on the strengths and weaknesses of each weapon class. Write down the score as well, if a scoring system has been implemented.
• When the hour is up, have everyone switch teams. Do the same with the new set of weapons.
• Start an impromptu discussion. Write down opinions, observations, and final scores.
• You should have a hypothesis now. Check with the lead to see if you have enough time to investi-gate your hypothesis. If the lead suggests it, arrange everyone into different task forces and test it. (Task forces will be discussed in more detail in Chapter 6.)
• Chances are, you now have a good idea of how each weapon feels. Communicate your findings to the lead in written form.

Once developers come up with a new build, you'll have to do this all over again. In Call of Duty 3, the producers tasked the team with playing a lot of Halo 2 at home for balancing. After this was done, the devs changed all weapon values in the game to Halo 2 levels. Gameplay became arcade-ish, with several hits before someone died. Once the entire team despised the new values, the devs interceded again and brought them closer to CoD levels--not quite like CoD2 but closer than Halo! The result was a game easier to play than CoD2, but not as forgiving as Halo--which made it fun. Ultimately, this is the purpose of all balance testing. If you see players describing your game as "fun" on the Internet, it's a job well done!

Compatibility Testing

Exclusive to PC, compatibility testing is the process by which the game is tested with multiple hardware configurations. The objective is to make sure the game is fully compatible with parts and peripherals found on the market. Testers need to be proficient in PC assembly and troubleshooting, since the job consists of installing multiple pieces of hardware and seeing how the game runs with each one of them. Compatibility testing is much more technically demanding than balance testing. Installing GPUs (graphics processing units, the high-performance video cards used for 3D games) and RAM modules is not for the faint of heart. Even background applications can have an impact on how a game runs. Knowing how to perform compatibility testing is a somewhat rare skill, and most testers will never need to learn it. However, in order to advance your career, you might have to tackle compatibility testing before moving on to other things.

Most games made by Blizzard (e.g., Diablo, Diablo II, World of Warcraft, StarCraft) are virtually bug-free because Blizzard does not mind pushing a launch back six months in order to present a polished product.

Floyd Billings, (Assistant QA Lead, Sony Online Entertainment)

Audio Compatibility Testing for Mobile Games

In particular, music and sound for cell phone games present a challenge because every phone has a different instrument set. What sounds good on one phone can sound horrible on another. Our phone game projects require testing on a lot of devices to ensure the greatest level of accuracy.

Eric Doggett (Owner, Moondog Media)

I am currently working on a 14-game project for one company. The games will be ported to over 15 different mobile phones, each with their own audio file types, size limits, and speakers. I had to audition each phone and build my own playback system comprising two 1/2 -inch speakers configured to play back directly from my workstation. When speakers are this small, much of the bass and treble are stripped away and the sound is extremely distorted. There is a lot of work involved in testing these devices and making them sound good!

Ben Long (Composer & Sound Designer, Noise Buffet)

Soaking

"Soaking" sounds like cooking--and in a way, it is. In game development, soaking is a sub-discipline of testing--one where the tester doesn't really do anything but leaves the console or computer running for extended lengths of time. For example, a tester might leave the game on the title screen for three days and then verify whether it crashed or not. Soaking is necessary because sometimes memory leaks or rounding errors might hurt the game's stability in the long run--which would be undetectable during normal testing. Soaking testing is usually done by a lead tester.

Compliance Testing

Compliance testing is the one testing variety everybody seems to hate. But what gives compliance testing such a bad rap? Before a game is sold in the market, it first needs to go though certification; this process is conducted by the hardware developers themselves (such as Sony, Microsoft or Nintendo), who need to be absolutely sure that a game follows established guidelines before it is allowed to be sold on store shelves:

• Sony has what is called TRC (Technical Requirements Checklist).
• Microsoft's is similar, but not the same: TCR (Technical Certification Requirements).
• Nintendo uses the same term it did for Super Mario Bros. in the 1980s: Lot Check. (See Chapter 1 for more details.)

Each compliance checklist is composed of several different basic categories, such as the warning that flashes on the screen if you don't have enough storage space. This warning must follow certain guidelines or the game will fail once it reaches the console manufacturer (which we usually refer to as "first party").

Nintendo has a really good reputation for having games that appear to be bug-free. Super Mario Galaxy on the Wii was not only a great game, but it also showed tremendous polish and a dedication to making sure a weird bug never took you out of the experience.

Josh Bear (Chief Creative Officer, Twisted Pixel Games)

RocketBowl Microsoft Certification

We developed RocketBowl for Xbox Live Arcade last year, and the testing phase was a grueling 10 weeks. Microsoft certification requirements are huge (250 pages of them), especially for multiplayer games. Even with all of the testing, we failed certification the first time (mostly due to translation issues) and had to resubmit. Overall, we learned a lot about how to make a solid product for XBLA (Xbox Live Arcade).

Justin Mette (President, 21-6 Productions)

Grand Theft Auto IV: The Rockstar of Compliance Testing

Rockstar's Grand Theft Auto IV was a very clean Format QA [another term for compliance testing] submission from the beginning. This high-profile title had its release date slip--which most likely contributed to the devel-oper's ability to ensure that bugs were minimized and that features were thoroughly tested in-house. Additionally, it should not be overlooked that the large budget most likely played a key role in ensuring that project management could devote necessary resources to do the work--and do it right. Finally, with a simultaneous multi-platform release date, this increased total headcount--and I'm positive more eyes were working the product as a result of this marketing strategy. In conclusion, it cannot be stressed enough that top notch software engineers and supportive project management undoubtedly played the single most important role in this title's tremendous success.

Jerome Strach (QA Manager, Sony Computer Entertainment America)

Other common compliance requirements:

• The game must not display any religious images
• The game should pause once the controllers are not attached
• Before formatting a memory card or a hard drive, a message warns the user that all the data will be erased
• All copyrighted brands or names are acompanied by their respective copyright or trademark signs
• If data gets corrupted, the game warns the user and suggests reformatting the memory card or hard drive

Testing for compliance gets repetitive quite easily. Can you imagine testing a PS2 memory card for two hours, following very specific instructions? There's simply no playing involved in something like this; it's hard work--that's all. Combing the game for these items is very difficult; very few testers are actually able to go though the hardware manufacturer's requirements and find all the compliance bugs. Compliance testing is a unique skill and one developers and publishers are known to seek out in new hires.

Compliance Testing for Gun

Luis was once selected among a large group of testers to learn compliance testing. This happened in the lull between two projects. Each one of the testers was provided with old builds (preliminary versions) of Gun and we would spend days going though the compliance checklists, trying to get the hang of it. While some of the items were fairly simple (e.g., "the game must go to pause mode once the controller is disconnected"), others could have been classified as "hardcore" (such as the PS2 network adaptor, which required a CD driver)!

Localization Testing

Localization testing is a reality because we live in a global economy. "Localization" involves converting a game from one region to another, and it usually includes translation. A game might be made in China with the help of Irish and Australian developers--or maybe it was made by someone in Los Angeles, but it was exported worldwide. More com-monly, games are made in Japan and Korea and end up on American shores. For example, Konami and Capcom are famous Japanese developers, and Gravity Interactive is a name brand in South Korea. Many of Nintendo's core games first see life in Japan, and then they are localized for North America. Once these games get exported overseas, they need to be translated. However, these translation jobs are sometimes full of mistakes. This is a bad sign for a foreign game, especially if the game in question is striving for some kind of dramatic impact.

On the PC side, Blizzard rules the roost on polish. Diablo, Warcraft, Starcraft, World of Warcraft: All of those games were polished to an incredible level for the PC.

Starr Long (Executive Producer, The Walt Disney Company)

Localization Testing in Call of Duty 2

Usually, games are tested once they arrive in the US, but Luis had to do localization testing on Call of Duty 2 once it went to China and Japan. His character would have to keep dying, over and over again, so that Luis could compare the "death quotes" on the screen with the correct Chinese or Japanese characters. This was pure torture--and he's glad he never had to do that again! Still, it's a job that must be done--and there's always somebody that enjoys this kind of task. Currently, Korean MMOs are the ones that must go through careful localization--since mainstream Japanese games rarely have this problem anymore.

All Your Base are Belong to Us": Zero Wing's Claim to Fame?

The most famous example of bad translation can be found in Zero Wing. "All Your Base Are Belong to Us"--a line from the intro of the Sega Genesis game--inspired a slew of media homages (including techno dance tracks, t-shirts, and April Fool's jokes). Even YouTube posted the phrase "ALL YOUR VIDEO ARE BELONG TO US" when it was taken down temporarily for maintenance.

As an example, here is the full dialogue seen in the game's intro: Narrator: In A.D. 2101, war was beginning.
Captain: What happen?
Mechanic: Somebody set up us the bomb.
Operator: We get signal.
Captain: What!
Operator: Main screen turn on.
Captain: It's you!!
CATS: How are you gentlemen!!
CATS: All your base are belong to us.
CATS: You are on the way to destruction.
Captain: What you say !!
CATS: You have no chance to survive make your time.
CATS: Ha Ha Ha Ha ....
Operator: Captain!!
Captain: Take off every 'ZIG' !!
Captain: You know what you doing.
Captain: Move 'ZIG'.
Captain: For great justice.

Playtesting The "fun testing" seen on television ads is playtesting--first mentioned in Chapter 2. Contrary to productivity software such as Microsoft Office, games need to be fun to play. To simply "work as intended" is not enough; game designers must tap into what is known as "the fun factor." In a Gamasutra article entitled "Secrets of the Sages: Level Design", The Levellord from Ritual Entertainment, explains:

"There are no defined rules for fun, and the only way to ensure the Fun Factor is to playtest. The easy part about adding the Fun Factor is that most all of us have the same concept of fun; that is, if you the game developer think it's fun, then the game audience is likely to think so, too. The Fun Factor is not transient or ephemeral, either. It should survive countless trials and tests and still be entertaining in the end. This is the only way to ensure that a game is fun--to play test it over and over."

Making a game fun to play takes a lot more than good intentions and magic dust. It takes guts, determination, and exemplary teamwork. Likewise, playtesting is not something you do to pass the time and have a couple of laughs. The best playtesters can divide themselves into two personas: the player, and the professional.

1. Player: Always ready for the next thrill
2. Professional: Always watchful of gameplay mechanics (e.g., navigation, aiming, targeting, interaction, behavior, physics, artificial intelligence, goals)

A game can only be great when playtesting is taken very seriously by everyone in the team.

Nintendo's "Fun Factor"

Lots of developers such as Blizzard and LucasArts demonstrate excellent quality assurance over the years, but Nintendo in particular seems to have taken control of this from the very beginning. Nintendo's games have always exhibited a fine gameplay balance, a "fun factor," that often proves elusive in the industry. This emphasis on balancing and polishing ga-meplay, by definition, leads to top-notch quality control. If you're focusing as hard as possible on making the game fun--and testing and retesting it--you're bound to uncover and address any significant bugs. Along the way, you'll gain enough control of the development process so that new bugs aren't introduced at a point where they may not be found later on.

Jamie Lendino (Composer & Sound Designer, Sound For Games Interactive)

Playtesting in Call of Duty 3: Get that Bike Right!

Playtesting involves going over all possible varieties of play and making sure all of them are fun. This is serious work: All it takes is for one faulty game sub-system to ruin the whole experience. We'll give an example of Call of Duty 3, a game in which Luis worked as a production tester. Call of Duty 3 changed the CoD series forever with the introduction of vehicles. Until CoD3, all the vehicles in a level were mere decoration--accessories for the level design. With CoD3, jeeps, bikes, and tanks became not only drivable but essential strategy elements.

However, adding vehicles to a first-person shooter (FPS) is more complex than it might seem at first. When you drive a jeep in CoD3, the last thing you want to be is a sitting duck. With a bike, you want to avoid becoming a sluggish moving target--and the tank must be maneuverable enough to cram through tight European villages such as Poissons.

When fine-tuning the bikes, Luis decided he'd do everything he could to make the bikes fun. (Luis is a racing game fanatic, so he figured he was the one for the job.) There were two different bikes for the US and Germany. At first, they were both quite slow--and their handling was sub par. In order for the bikes to be effective strategically, they first needed to be drivable--offering a degree of control to the rider. Bike crashes in real life are deadly, but bike crashes in-game can bring defeat and certain humiliation.

As a production tester, Luis would spend a lot of time driving the bikes fast, jumping, hitting obstacles, and attempting to run his enemies over. When a big problem was spotted, he'd write to the producers in detail--explaining why the current suspension setup was ineffective or "boring." Soon enough, the programmers made the bikes faster. Later on, they made drifting possible--with a tail-happy rear-end. These two touches of gameplay allowed the bikes to be effective and fun, pushing the fun factor way up.

Usability Testing

Usability testing is a fairly recent advance in software testing--surprisingly so, considering its importance in the player's satisfaction. Microsoft is a big fan of usability testing--having learned the hard way after users kept complaining about how Windows machines were so much harder to use than a similar Macintosh system.

Usable means "easy to use"--not "easy" as in difficulty level, but intuitive to interact with other characters, use items, find your way, or drive vehicles. If a certain mechanism in the game makes no sense at all (e.g., you can't aim and move at the same time), this could be written as a usability bug. Perhaps the game was designed that way and nothing is really broken. It doesn't matter: Gameplay mechanisms need to make sense and be easy to use. You should worry about beating the game, not the control scheme. Energy spent otherwise takes away from the fun you might be having otherwise. Since it is responsible for visual control and feedback schemes, a game's visual interface is scrutinized during the usability testing process. Although uncommon for QA (which is used to catch more superficial bugs), all testers end up having to deal with usability issues. You may have even had to deal with them yourself--in games you've purchased!

We've discussed several areas of knowledge within testing that are considered "disciplines"--including balance, com-patibility, compliance, localization, playtesting, and usability testing. Understanding these disciplines will help you become a well-rounded tester. As you become more skilled as a tester, you will be able to apply advanced techniques (discussed in Chapter 6) to each of these disciplines.

As you can see, game testing is many times more complex than the brainless ads on television might suggest. While testing a game is a skill easily learned, understanding the game industry and how it relates to each variety of testing is a much bigger challenge. Where QA testing has a "brute force" aspect to it (after all, you're banking on a large number of bodies), production testing might be almost Jedi-like--with a couple of ace testers that are the only ones capable of cracking tough networking bugs. In the next chapter, we'll learn what steps need to be taken before you begin testing.

Chapter Review

1. Play a game for at least 4 hours--taking notes on the various bugs you find. Categorize each bug with an appropriate severity level (low, medium, high, or critical). What sorts of bugs seem to be higher priority than others? Is there a pattern associated with severity level (e.g., location in the game, discipline, frequency)?
2. Go to BetaWatcher.com and join an open beta test for an upcoming game. Maintain a journal during your testing experience and report your weekly progress. When the test has completed, discuss what you learned during the process.
3. What are the testing varieties discussed in this chapter, and how do they differ? Selecting one of these varieties, and then test an existing game based on that variety's criteria. Provide a list of im-provements that should be made to this game based on the variety you chose.