Here are some general rules I’ve noted while testing Vikings Gone Wild and Heroes of Paragon.

 I am writing this post with mobile f2p games in mind, but you can apply most of these notes to any other kind of game you’re testing.

  1. Start early.

    This is something we’ve learned the hard way with Heroes of Paragon.
    In the early stages we were like “Meh, let’s wait for this thing to be a little bit more developed, so we can actually, seriously test all of the features”. This mindset lasted a few months. Well, spoiler alert, this does not work. Towards the end, as everyone is chasing the deadline, there’s constantly less time for both testing and fixing. With some features you need an active help from the developers and it can happen when they simply do not have much time for this. This lowers the end-product quality quite drastically. So do not wait. Fight to get your hands on the game as soon as possible.

  2. Automation is the key.

    Especially for Android. It is important also for iOS, but in its case the range of devices isn’t nearly as big. And you probably don’t care too much about the first generations of iPhones. So in the worst case scenario you have around 15-20 devices to cover. I wouldn’t be thrilled by that perspective, but in most cases it would be doable. If you can afford to get them, that is.
    For Android it’s not that simple. There are hundreds or devices, part of them with custom OS. There is absolutely no way you can cover even 25% of them manually.
    There are a few companies who allow you to test on real devices remotely, such as AppThwackDeviceAnywhereXamarin and the one I use myself – Testdroid.
    It’s especially crucial to automate the first stages of the game on as many devices as you can. Why is it so important? Simply put – players on early stages are not attached to your game at all. If they find a glitch, a crash or any other noticeable and annoying bug – they will almost certainly leave the game and, in most cases, never come back. If your time and/or resources for this are limited – focus on early stage automation. Having the game tested od 50+ or even sometimes 100+ real devices is a huge potential boost to quality. You see every major graphical glitch and for every crash you get a log file, which is priceless for your developer team.
    A good practice is also having the FPS meter displayed in-game, so on screenshots you can also monitor the performance, if there are no tools available for it.
    I will be sharing exactly how I use automation to test the tutorial phase in Heroes of Paragon in another post.

  3. Soak-test. 

    In F2P games, many players will play really long sessions to gain advantage over less-active opponents. Make sure the game doesn’t slow down or crash after you play 40 minutes or longer. You can even leave a device with the game open overnight and see if it’s still there in the morning (the game, not the device. If the device is not there, call the police, not the programmers). Unfortunately, it’s difficult to gather logs from this type of crash, as for Android it would take probably a few gigabytes. For iOS it’s simpler (I didn’t think I’d ever say this!), as crash logs are saved automatically on the device (usually…).

  4. Think about low-end Android.

    On the market, there are a lot of cheap Android devices, that are extremely popular, but unfortunately not too exciting in terms of performance. You have to think about those. Automation is super helpful for this, but it’s very important to have at least one device like this in the office, to see the performance directly. Provide feedback and log files to your developers to get the best performance possible on those devices. If it’s impossible to assure proper playability, talk to the Product Manager about excluding those devices, simply not making the game available on them. If the game is not playable, you don’t want people to get frustrated, go to the App Store and give you 1 star and a mean comment. It’s better if they are not able to install it at all.

  5. Compare.

    Feel free to download the top-grossing games of similar type and see how they perform. Compare the general UX.

  6. Reach out to players.

    If you have a possibility to have external beta testers – don’t hesitate, go for it. No matter how good a tester you are, there is no way you will cover the game the same way 10 or 20 regular players will. And you also get feedback on playability, game balance and the infamous fun factor.
    For both our games we have now an active forums, which can give you a lot of help in polishing the game at a very little cost.
    Also – check out the comments under your game in the stores – look for recurring issues, try to reproduce them. The problem with it is that most of the comments are very non-specific and don’t give you a lot of detail.

  7. Plan ahead. Do not waste time on trying to be perfect.

    I am the only tester in EVERYDAYiPLAY, responsible for quality of now two, and soon three games at once. Not wasting time is crucial for me.
    The sooner you can get the design doc of a feature you’re about to test, the better.
    For games that are already live, a lot of your time will be allocated for sanity-checks or regression testing. Do not overthink it. If there were no changes in a specific functionality, do not waste too much of your time on it, most of the time there is no need to go deep. And there is usually not much time.
    Make a list of key features that are absolutely crucial to work perfectly, such as player creation, payments, global chat, graphical settings and so on. You get the idea.
    Always be 100% thorough on those. If a functionality was changed during a sprint, temporarily add it to this critical list. For others – just do a quick check-up to make sure it’s there and nothing broke in a spectacular way.
    Another great way to save some time on regression is the above-mentioned automation.

  8. Work closely with the Game Designer.

    As a QA Engineer/Specialist/Wizard/Acrobat/Whatevertheywantyoutobe, making sure the game is not crashing is not your only responsibility. People often forget about this.
    QA stands for Quality Assurance. And a massive part of game’s quality is its balance and so-called fun factor. We do not only want games to work correctly, we want them to be enjoyable. Look at it from a bussiness perspective – in F2P, you don’t get paid just because someone got your game on their phone. You have to make sure they have fun, they return to the game, preferably on a daily basis, and – if they’re kind enough – spend their money on it. They won’t do it if the game is boring, confusing, too hard/too easy and so on. Help your Game Designer find the sweet spot.

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someoneShare on Reddit