Designing a Gesture Input System – Part 2

When we were done coding the algorithms for recognizing user gestures, we faced a big problem: gibberish input. Machine learning algorithms like the one we use are trained to recognize a given number of things. Our neural network assumes that whatever you draw is one of the spell shapes that the network was trained with.

To give an example: you may draw the number 8, and if this shape was not part of the training set of shapes, the network may end up being convinced that you just drew a circle. Because out of all the shapes the network learned, a circle may be the closest shape to the one you drew. In order to solve this, we had to implement a rejection mechanism that detects unknown shapes.

This is essentially a separate filter that’s applied after the neural network outputs its prediction. We run a few tests figuring out whether the network’s guess makes sense given the user input, and if it does not we tell the player that their spell attempt failed. Getting back to the example from before, the network would tell us that the player drew a circle. We would run our filters and find out that the 8 that was drawn is too different from a circle (the closest known shape) and fail the spell.

We find machine learning a fascinating concept in general, and since our villains are supposed to be very powerful iconic characters, we may use machine learning algorithms for enemy AI, too. This would allow enemies to improve during the course of the game, adapting to the user’s play style. No promises on this one though, sometimes “faking it” ends up being the better choice, time will tell. And by the way, a bit of “faking it” can’t ever be avoided, because an AI that adapts perfectly to the player would have to be dumbed down in order to be beatable at all.


Leave a comment