Natural interfaces for therapeutic purposes

In recent years, the world of computers and telecommunications has experienced an increase in the presence and popularity of the so-called Natural User Interfaces (NUI). Its mission is to replace the usual interfaces of a computer system such as the mouse and keyboard by others more adapted to the human body, i.e. voice recognition, motion detection,  or face and emotion recognition.

This technology is more and more appreciated, since it offers a more intuitive handling of the technological tools, resulting primarily in being a great help for people with disabilities. In this sense, one of the research topics of GAMMA focuses on the development of applications to be used preferably by people with reduced mobility. Here, a motion capture camera is involved as the main input device. This kind of camera achieved much popularity in the video games industry, e.g. to play sports games, but has failed to maintain the user’s interest. However, there are multiple possibilities in the medical world, not only on medical side, also for patients, i.e. the accessibility to computer programs can be improved or attractive virtual environments can be created for doing physical exercises (exergames).

In relation to the latter, we are developing two projects which have a broad perspective: MoKey (Motion Keyboard) and Blexer (Blender Exergames). 

MoKey is a NUI designed and developed to control any kind of computer software through body movements. Movements are translated to key strokes, such that programs can be made accessible to users with reduced motor functions. MoKey can also be applied for daily exercises and rehabilitation, by configuring it to the movements a patient needs to perform. In this way, the exercises can be performed during moments of leisure (i.e. a game, an e-book reader) or at work (presentation, telephone conferencing etc.).


                                                                Main window of Mokeyy is to assign movements to key strokes.

Blexer is a system that integrates different video games controlled by body movements. The aim is to create an environment that gives access to different types of video games can be adapted to users with reduced mobility. The games should be used to combine entertainment with necessary daily physical exercises. Here, the priority objective is to achieve maximum immersion of the users in such a way that they are not aware of doing physical exercises. Therefore, gamification methods and artificial intelligence are combined to achieve individual adaptation and personalization. At the same time, the system is in constant communication with a doctor or therapist via the Blexer-med medical platform, for monitoring the progress of the patients and adjusting the game parameters to their individual needs.

Complete Blexer system Left: user's side. Right: medical side.

Different exercises within the "Phiby s Adventures" video game.