All playtesters lie. Every single one of them, including you. Get used to it.
Okay, lie is probably a harsh word, because for the most part they don’t mean to mislead you and give you biased feedback (although some definitely do). However, no matter how pure the intentions are, humans are biased creatures, and there is no getting around it.
So you’ve written a game and now you’re ready to play and get all that “honest” feedback about what works, what’s fun, what’s not, balance issues, and all the ways the game breaks down. You play and hear a variety of feedback from yourself and everyone else.
The first problem you will run into is that this feedback will all be different. Only once did I get the same feedback from everyone, and that was to tell me the game I created was truly horrible, unplayable and unfixable. The only other time that you’ll get a consensus in your feedback is if everyone consciously lies to you to make you feel better, resulting in absolutely nothing useful. In other words, unified feedback is probably not what you’re hoping for, ever. So while it’s a problem to get a wide variety of feedback, it’s actually what you need.
Now comes the tricky part, figuring out exactly in what ways people are biased. THIS is truly the single most important skill you will develop in the playtest process, and the hardest. However, until you know a little bit about preferences, background, play style, and dislikes of a playtester, it’s incredibly hard to really use the feedback they give you.
This becomes even more obvious when playtesting for a large, complex and many faceted game, like Malifaux. That also means Malifaux provides plenty of examples of exactly what I mean.
John (name changed) was on my original team developing Malifaux, so he endured all the ups and downs of the first few drafts of the game. John was also incredibly good at identifying breaks in a game and when things would just simply not work as written. That said, he was also prone to calling me up and informing me that the entire game was hopelessly broken and unplayable. Only once do I remember him bringing up an actual unfixable problem which required a major shift in the rules to correct. The other times, the fix required changing one or two words, or adjusting a number here or there. As a result, I got to a place that whenever he cried “broken” to me, my first question was “Is it actually broken, or is it John-broken.” I actually always knew that the answer was that it was, in fact, only John-broken, and something that would take 2 minutes to fix.
Faction bias also is one which is inescapable. Everyone has the faction or masters they play in Malifaux, and those faction and masters that their opponents most often play that they dislike. Try as they might, without fail, their feedback will always try to make their faction too strong, and the faction they dislike too weak. In fact, during playtest for Malifaux second edition, every single playtester had to be classified by which faction they most liked, and which they disliked most, in order to compensate for their bias.
It is true however, that some playtesters are aware of their bias, and try to compensate in order to provide better feedback. This is actually often times even worse! I’ll use myself as an example. My favorite Malifaux master during development was Marcus, and as such I didn’t want to be that guy who made my favorite master far and away the best in the game. What was the result? My involvement in playtesting Marcus and trying to compensate for my bias left Marcus as easily one of the worst masters in the game. Good job me!
So there are two lessons here. If you are offering up one of your games to playtest, be aware of what you’ll deal with. You’ll need to both listen to the feedback but also understand in what ways that playtester will be lying to you in their feedback. Just expect it, it’s not their fault.
The other lesson is that if you are a playtester, understand that no matter how hard you try, your feedback will be inaccurate, there is no getting around it. In fact, the more you try to get around it, the less useful it will probably become. So give your honest feedback to the playtest, and then maybe take a second to evaluate exactly what your biases may be and let those be known as well. Also be understanding when your feedback does not result in every change you wanted, because that would actually likely be terrible for the game itself.
Ok, so it’s not really lying, but in my years of game design, never once have I had a playtester (including myself), who ever gave completely accurate feedback.
Two related topics you may choose to look up yourself until I cover them in more detail. First, certain professionals train for years to understand desires and biases. It takes skill and a specific mindset to objectively evaluate feedback and to understand how people communicate desires and preferences. These fields include marketing and advertising or usability testing. Second, group psychology. When feedback from multiple playtesting groups is available to all (such as in a forum where everyone has access to everyone else’s feedback), the feedback can be subject to some dangerous pitfalls (dangerous for the quality of the game). Oftentimes the most vocal perspectives are at the extremes and are loudest and most persistent, drowning out the voices of the majority. Also, it is unfortunate but many people can be easily swayed by a loud, persistent, often times minority and completely bias, opinion.