Testing Ambassadors: How we Supercharged our Internal QA
By Mateja Dukarič, Quality Engineer and Testing Ambassador

When your games reach billions of people, you can’t afford to release a buggy product. Out there, in the wild, every little coding mishap and every slightly annoying design flaw will pop up millions of times. That’s why we need to test!

As they grow, gaming companies often scale their testing capabilities by outsourcing QA. At Outfit7, we took a different approach and got everyone in the company involved in testing. We did it by creating a culture of testing with the help of Testing Ambassadors.

Since our products are mobile games, this really wasn’t hard to do. Here, testing isn’t a chore, it’s fun! Everyone loves taking some time out of their day to play our games, especially since we know we’re helping our QA team (and our users) out by doing it! We actually have a strong internal QA team that makes sure the quality of our games is pumped as close to 100% as possible on every single project. But even the most dedicated Quality Engineers can’t test every scenario on every device! That’s where the rest of us come in.

When we were a one-game startup, getting everyone to test our game was easy. As Outfit7 grew, both in terms of team size, games published and locations around the world, this got a bit more tricky. At a certain point, we realized we needed to formalize internal testing if we want it to be efficient and produce results.

Maybe this has you thinking of people in suits testing games and filling out forms about it. Don’t be fooled, “formalized” for us just means that every team (or sometimes every floor in an office) has a designated Testing Ambassador who gets instructions about tests from QA and then gets the team to play together. That’s as formal as it gets around these parts, and we make sure that testing never feels like work.

We love it when testing becomes an event in itself. Sometimes, the Ambassadors would jokingly bribe us with candy and it becomes a sort of like a real-life minigame with sugary rewards. Other times, we dress up for the occasion – like when we transform ourselves into unicorns to test an especially magical My Talking Angela update!

In the beginning, we did it without much structure. Anyone could give any feedback they want in any form they wanted. Over time, we’ve figured out what kind of feedback can help our product team the most. So now Testing Ambassadors are responsible for giving out instructions, sending out feedback templates, and collecting the information in an organized form. And, of course, making sure the entire process stays fun for everyone involved!

We take all feedback seriously and try to implement as much as possible. Our team always goes through what everyone has to say, but the reality of it is not all fixes make it to production. We get so much feedback, we have to triage it: what gets done now, what gets put on the roadmap and what gets pushed aside.

We don’t just test for bugs, we play the games to see if all the instructions are clear, if the features make sense and if they’re actually fun. For example, the first time we tested My Talking Hank, no one could understand the tutorial. So we completely changed it for the actual release version. 

Another example would be Angela’s makeup in My Talking Angela. During testing, some of our teammates mentioned that putting on different types of makeup at once can produce results that are… not exactly flattering. This was definitely one of the more memorable pieces of feedback we got. One of our teammates sent a photo of Angela in ridiculous makeup with the caption: “Here’s a nice photo of Angela to haunt your dreams!”

But the thing is, one person’s cringe moment could be another’s favorite part of the game! We knew that the makeup feature could produce ridiculous results, but we also knew a lot of people would get a real kick out of it. Beauty is in the eye of the beholder, and so is fun! It’s up to our QA and product teams to balance out all the different opinions we get from testing.

There’s a lot of diversity in the types of gamers in our company. So getting everyone to test gives us a wide perspective since a casual gamer will usually spot something completely different than a hardcore one. It’s also interesting how testing always reveals how differently people from different departments think, in general. 

The feedback from the back-end guys is completely different than from, let’s say, marketing! For example, the back-end team thought it was weird that washing Angela’s face doesn’t remove her makeup. Marketing, on the other hand, immediately noticed that there’s a “makeup pad” there especially for that task, and wasn’t confused by the concept of water-resistant makeup. Guess which department has more female teammates?

Besides getting different people to bring their unique perspectives, getting everyone involved also allows us to test our games on a lot more different devices and OS setups than our QA team could emulate, which is great for covering a range of scenarios.

It sounds like a lot of work, but because we organized our Ambassadors well, this all runs on minimal effort. Or maybe it just feels that way, because we’re essentially playing games on the job! Each employee spends an average of half an hour per week testing and we currently have 43 Ambassadors making it happen at our offices around the world.

Getting the whole company involved in testing has become an important part of our work as well as our culture. It’s something that keeps everyone connected to our core products and gives all of us a chance to have our voice heard. Besides, when your core product is fun, testing it with your teammates is something you look forward to.