A few of us from the Cambridge NetOS Group were involved in the Cambridge Science Festival last weekend. We set up a demo of EmotionSense: a mobile tool for social psychology research that attempts to detect the emotion of mobile phone users: the system works by recording samples of the noise around the phone and using machine learning techniques to match speakers’ emotions to a library of actors.

Since we were expecting mostly kids to show up, we changed the system to be a game instead of an emotion-detector. The aim of the game was to “convince the phone” that you were speaking with a given emotion. There were up to 6 points available, and the levels were: silent, neutral, sad, happy, angry, and afraid. Five seconds to express your emotion in each level, one point for each correct classification. The children turned out to be very competitive and so I set up a leader board (photo) for them to proudly add their name to. Chloë, Kiran (photo) and I were kept busy for most of the day.

One of the research questions that we are tackling is improving the accuracy of the system - we would like it to better match the sound of people speaking to the emotions they are trying to express. A notable problem in this domain is the lack of labelled datasets that match features of human voices to emotions. The science fair gave us a great opportunity to collect a sample of anonymised data (note: we didn’t keep any recordings of peoples’ voices!) of people trying to express particular emotions. We have yet to look into the results - but it seems we collected a good amount of data to play with.

It was interesting that children turned out to be better at the game than adults. In fact, adults would attempt to “express” an emotion using words rather than intonations (e.g., “I’m so happy!”), while children would make the sounds that the system was looking for. Finally, here is a short video showing Chloë (video) in action with two young kids. These kids actually were the first to get to 6 points (and what I didn’t catch was the ensuing fight over who gets to write their name on the leader board first!). The video is worth watching since (nobody noticed) but Kiran nearly chucked a glass of water over Chloë near the end - epic save.