Lesson 18 · Part 5: Making It

Playtesting

How to watch other people play your game, what to listen for, and which kinds of testing are worth your time.

12 min read

Playtesting

You’ve made a little card game, and the very first thing a player needs to do is flip over the top card of the deck. You put the deck right in the middle of the table. You even drew an arrow on it.

You hand the game to a friend. They read the rule sheet. They shuffle their hand. They arrange their tokens into a neat row and ask whether it’s their turn. They never touch the deck. You try a second friend, and a third, and by the fifth you’ve watched five intelligent adults ignore the most obvious thing you’ve ever made.

Every game designer has a version of this story, and every one of them will tell you it never stops being frustrating. It’s also the only reliable way to find out what you’ve actually built.

A peek inside someone else’s head

A playtest is a session where someone who isn’t you plays your game while you watch. Its purpose goes back to Level 3. A game really lives in the player’s mental model, the picture in their head of how it works. Until you test, the only model you can see is your own, and yours was built by someone who already knows all the answers. You can’t un-know them.

Writers know this problem. You can reread your own essay ten times and never spot the paragraph that makes no sense, because your brain fills in what you meant. A fresh reader trips over it in seconds. Recipe testers know it too. Cookbook publishers hand recipes to home cooks who’ve never seen them, precisely because the chef who wrote “fold gently” knows what that means and nobody else does.

Testing doesn’t wait until the game is finished. It starts with you poking at your own first prototypes. A little later you bring in a few outsiders to see whether they can figure out how to play. Later still you might bring in larger groups, and with a computer game, eventually thousands of players whose paths through the game can be counted and charted.

Fresh people, in pairs, in silence

Over the years, designers have worked out a way of running these sessions, and each part of it fixes a specific problem.

Problem one: a tester can only be new once. Once someone knows where the deck is, they can never again show you what a newcomer sees. So you watch a tester once and don’t use them again. Will Wright’s old studio called this the Kleenex test, because you use each tester once and throw them away (figuratively, please). After every round of fixes, a brand-new batch of people comes in who’ve never seen the game. Comedians do a version of this. A joke that’s been heard once is useless for testing, so they need a fresh crowd every night.

Problem two: people alone go quiet. You can see what a solo tester does, but you’ve no idea what they’re thinking. The fix is paired testing: sit two people who know each other, friends or a couple or siblings, in front of the game together. They’ll talk nonstop. “Flip that one.” “No, that’s the discard pile.” “Then what’s the arrow for?” You’re hearing their mental models out loud, for free.

Problem three: you. This is the hard one. You give the testers one sentence (“it’s a game about running a bakery”), set it up, and then you say nothing. You don’t help. You don’t explain. You don’t lean in and point at the deck, even when every cell in your body wants to. You sit a little behind them, listen, and write down what they say, especially the wrong guesses. If several designers are watching, they don’t compare notes until it’s over, so nobody’s opinion leaks into anybody else’s.

Afterward, look for the two or three problems that came up again and again. Those go to the top of your risk triage list from Level 17. Fix them, bring in new testers, and check. If your fixes worked, the next group will hit three different problems, and around you go again.

The problem under the request

Testers will tell you things, and most of what they tell you needs translating.

People describe solutions. A player says the game would be amazing with a jetpack and a laser beam. What actually happened is that they were stuck at the bottom of a cliff and wanted to get up it. That’s the gap between solutions and problems, and Wright is refreshingly blunt about it:

“Probably their solutions suck, but the problem they’re referencing doesn’t. That problem is real.”

Will Wright

Players aren’t designers and shouldn’t have to be. Sports coaches see this constantly. A player says “I need new boots.” What they mean is that they keep slipping on the turn, and the fix might be footwork, not footwear. When someone asks for a feature, ask yourself what wall they hit that made them wish for it.

Then there’s the question of whose complaints to believe. Level 9 introduced signal and noise, and it applies here in two ways. Across testers, noise is scattered: everybody grumbles about something different. When five separate pairs all miss the deck and agree on nothing else, the deck is signal. Across watchers it works too. One person may blame a muddle on the rule sheet and another on the board layout. When every watcher reads the results as pointing at the same cause, you can trust it.

A request to add something often hides a need to take something away. Testers ask for more when the real trouble is that there’s already too much competing for their attention. There’s an old saying, often linked to Japanese gardening, that a garden isn’t finished until there’s nothing left you can remove. Call the habit subtraction, and try it before you try adding.

Is it fun yet?

Early tests can’t tell you whether your game is fun, so don’t ask them to. Finding out that people can work the windshield wipers tells you nothing about how the car drives. Once enough parts work together, though, you start looking for engagement, meaning how strongly the game holds someone’s attention and makes them want to keep going.

Engagement covers a wide range. With a rough prototype, the struggle is keeping a tester at the table. With a nearly finished game that’s really working, the struggle is getting them away from it. Watch for a few plain signs. If you’re showing someone a game and they reach over and say “Ooh, let me try,” they’ve understood enough to want something. The best sign of all is a friend who says “one more round” three rounds in a row.

Engagement doesn’t require challenge, either. Some people are happiest messing about at their own speed, and others want to be pushed every minute. Letting them choose is the player-paced difficulty of Level 5, and it’s still the safest way to keep both kinds near flow.

With a big digital game, you can go beyond impressions. Metrics are measurements collected automatically from lots of players: how long they spend in each part, how many fail a challenge on the first try, the second, the third. That’s far better than watching ten people and trusting your gut. One caution: different player types produce very different numbers, so an average across all of them can describe nobody at all.

Who’s it for, and how would you know?

Testing also shows you your target audience, the group of people a game is mainly for. Start with a wide mix of testers and narrow down as you see who lights up. Be ready to be surprised, because predictions about who’ll love a game are often wrong. And be ready to follow what you see.

That points to one method designers often distrust. A focus group gathers people to react to a description of something that doesn’t exist yet. Wright’s studio once described four game ideas to a room of players, and the one they all hated was The Sims. The trouble is that a description forces listeners to design the game in their heads, and the game they imagine may be awful. When the same kind of people got to actually play it, they loved it. A focus group asks people to do the designer’s job. A playtest just asks them to play.

Last comes the beta test. A nearly complete game goes out to a big group, mostly to confirm it works at scale, catch the places people get stuck, and see real proportions. Five testers going one way and four the other might turn into five thousand against one thousand. Now and then a beta shows players doing something you never planned that turns out to be great, and you get to lean into it.

Try it · Pocket Game, step 18Shut up and watch

Find two people who know each other and have never seen your Pocket Game. Put the pieces in front of them with your five-line rules from Level 3 and one sentence about what the game is. Then stop talking. Don’t explain, correct, or hint, even when it hurts. Sit slightly behind them and write down what they say to each other, especially the wrong guesses. When they’re done, ask one question: “What were you trying to do?” Afterward, circle the three biggest problems you saw. For each one, write down the problem, not the fix they suggested.

Carry forward: Watch new players in silence, and fix the problem they had instead of the feature they asked for.