Emotional input

Some time ago, I was thinking of a problem that Aubrey touches on in one of his recent comments: how can you make emotional input emotional?

It’s not that difficult to create an intellectual system for inputting emotions. Say, the mood-o-meter: select ‘happy’ from a menu to indicate you’re happy. It’s been tried a couple of times in the history of mankind. (So when I say it’s not difficult, I mean one can sort of see where to go, even though nobody has done a good job of it yet and you’d probably have a really hard time convincing any company to invest money in it. But anyway, that’s a different entry.)

So how can you make emotional input emotional? How can you make a half-way decent attempt at capturing the honest emotion of the player? I came up with something that’s obviously flawed, but that I nevertheless find fascinating. Imagine a dialog screen where you see two people facing each other from the side. Imagine you’re controlling the person on the right. You’re using some controls, the arrow keys or something, to select phrases from a list. Normal conversation interface stuff. At the same time, you’re using the X axis of the mouse or an analogue stick to control your conversation ‘stance’. Basically, we’re looking at conversations as power struggles. You can either lean forwards to become more pushy and domineering, or backwards to give in to the other, or try to stay centered. (The other person’s movements might affect your stance, so there could be a balancing element. You could also add some simple resource management, to simulate poise or energy. I never developed the idea very far.) You would of course see the stances of both characters: they would lean forward and backward, changing their posture, etc.

The core idea is that an analogue, short feedback loop input mechanism should allow for very intuitive emotional input. A secondary, but crucial idea is that a horizontal physical movement is mapped to something that also intuitively feels horizontal.

Obvious flaws are that there’s nothing really logical to map to the Y axis, and a dialog system entirely based around domination, while promising, is also gimmicky and not generally usable.

Dialog systems, now there‘s a topic for a million blog entries…

I think Phil Harrison once mentioned that the PS3 should be capable of more advanced image analysis than the PS2, and that it might be possible to detect emotions, so maybe we’ll find another solution in a few years.

Comments 7

  1. Aubrey wrote:

    Elaborating on my comment the other day, I woke up this morning thinking that maybe a good term for the verbs that the player can use to imply their mood in the game world could be “emotional prosthetics”. Yeah, I know, I know. Buzzword alert. But think about it: for the time being, player’s bandwidth into the game is taken up very much by the base functionality they’re given: The moving, shooting, grabbing, and generic “use” functions that are used time and time again. There’s very little interface real-estate left for the player to imply and communicate their emotional state to the game (Eyetoy emotional recognition of the future not withstanding)… unless you want to create an entirely new mode of interface, which always strikes me as kind of inelegant.

    Some of these actions imply very obvious emotions. Equally, other actions have ambiguous emotions associated. Moving “sneakily” implies certain emotions when you’re “sneaking” past characters important in the story, but at some point, you’re just trying to get around, and not worrying too much about how people feel about how you do it!

    The most elegant solution I can think of (though certainly, this is also flawed) is to pick a wide range of actions to give the player, all of which have emotions which are very clearly realted to those actions. In essence, we give a wide spectrum of both actions and emotions associated with those actions – each tool in the box has two uses: the purely mechanical (moving around, affecting health, points, stats, whatever), and the emotionally expressive (which is arguably just mechanical, but is interacting with the “emotional game systems” of NPCs or other players or whatnot). Parrallel systems being affected by the same player input stimulus. Not a new concept – it’s really the core of “interesting choices” as JJ would have them.

    In very simplistic terms, if you want to display a negative stance, you do something that is considered “nasty” to an NPC (or whoever). If you want to show happy happy joy, you do something nice – give an addict some heroine, and he’ll love you for a day.

    I know it’s simple, but it saves giving the player an entirely new interface to have to deal with. The interface for game world manipulation, and emotional manipulation is unified.

    Where actions’ related emotions are ambigous, one can use surrounding context to define the intent of the action: given the command “grab”, and the context of a roof top, the “grab” is likely to be used in a threatening fashion if used on an NPC – threatening to throw them down to the street below. If that NPC is, however, hanging on to the edge of a building for dear life, grabbing becomes a positive thing.

    In a bar, “grabbing” a nice lady on the dance floor gives a positive connaitation: “I wanna dance with you, lady!”. “Grabbing” a lady away from her dancing partner (another context) has a bunch of other associated emotions flaring up in the lady and her jilted dance partner – all quantifyable things that the game system can be designed to recognize, deal with, and respond to.

    This context, in the case of actions implying ambiguous emotions, is really like the adverb, or the ‘stance’ in your example.

    I don’t think my solution gives quite as much control as yours, but at the same time, given enough contextual data, one can get a pretty decent facsimilie of the emotions that should be implied. For the most part, human behaviour is fairly deterministic – we just never seem to notice all the compound information that informs our decisions. Because, in the game, we create the context, it’s not too hard to guess at how the player would *want* to act. Actually, scrap that last paragraph – that’s like a really bad justification of emtioneering (*spit, spit*).

    But yeah. Don’t get me started on Dialogue systems – I could lose my job for saying too much.

    Posted 13 Jan 2005 at 11:23
  2. Jurie Horneman wrote:

    OK, there’s way more there than I can respond to now. One question, though: who’s JJ? To the best of my knowledge, the originator of the idea of interesting choices as a key element of interactivity is Sid Meier, with Noah Falstein being one of the biggest champions in the last decade or so.

    Posted 13 Jan 2005 at 11:40
  3. Aubrey wrote:

    Dr. Jesper Juul, ofcourse! You’re right about the sources, but Jesper did a talk on the subject a while back, basically saying that even though interesting choices a good, some people, or some situations prefer uninteresting choices. People sometimes like to think that they’re making the “right” choice. Games like DDR and Dragons Lair, and most linear games’ popularity make this clear. Some people don’t want to have to put up with the negative response to an interesting choice.

    Think of a guy who goes into a Starbucks and asks for a coffee, and the clerk says “Cappacino, frappacino, blah blah blah”, and the guy completely loses it and flips out like a total ninja screaming “I JUST WANT A COFFEE!!”.

    Posted 13 Jan 2005 at 12:06
  4. Aubrey wrote:

    Another thought: is it really important for the game to “know” how a player feels about something, if the player already has a way to express how they feel in the game?

    An example: As much as a player may be disgusted by mysogeny, they may want to role-play as a woman hater, just to see how the game responds.

    By this I mean, it’s more important for the game to respond to the actual input of the player, rather than second guessing how they feel and reacting to that hunch. If the system could get it wrong, making assumptions about unquantifyable data, it could back fire in a big way. But if the system is responding to how the player represents and conducts him/herself within the game world (which may or may not be a natural extension of their ego), no-one can really critisize the results, because we’re playing by acknowledged rules which are based on visible information.

    Posted 13 Jan 2005 at 13:23
  5. Noah Falstein wrote:


    I think the problems with emotional input are the highly subjective nature of emotions, and the huge amount of importance we (as social animals) put on the perception and interpretation of emotions. Aubrey, I think you are right about backfires. To take your example of the person at Starbucks, to him the choice was NOT interesting, but rather frustrating – I’ve felt the same way in RPG’s that go into endless detail about weapons – “would that be a Flemish blade with curved crossguard or the Viking blade with lenticular cross-section? I JUST WANT A SWORD”

    My point is that we are so well-equipped to sense emotional intention from a wide range of inputs – actual words, tone of voice, facial expression, social context, etc. – that trying to accept similar subtle input and have the computer interpret it is very likely to fall woefully short. It is the same reason that it is so hard to do convincing CG human faces, we’re just hard-wired to be very discriminating and solutions that address 2% of the problem just aren’t satisfying.

    Posted 13 Jan 2005 at 20:37
  6. Jurie Horneman wrote:

    Well…. I think the success of the movie and animation industry shows that people are fairly good at evoking and displaying emotions.

    I am not disagreeing with the point on too much choice per se, but I don’t see its relevance here.

    Finally, I was not talking about interpreting subtle input (apart from the brief aside on the PS3), but about making a very explicit input of emotion (think a 1D mood-o-meter) more intuitive and less intellectual. Display the two characters in an exaggerated toon style (say, Ren and Stimpy-ish), and I think it should work pretty well. Yes, there is more choice than a simple phrase menu, but I fail to see how this is wrong by itself.

    Of course, this all assumes that one can make some kind of interesting use of emotional input.

    Posted 13 Jan 2005 at 21:38
  7. Aubrey wrote:

    Indeed. You’d want some sort of concurrent emotional manipulation game system going on, which somehow feeds back into the rest of the game in a useful and interesting way.

    Good point, Noah. I hadn’t bothered to consider the subjectivity of “Interesting Choices”, but you’re right. It’s right there.

    Posted 14 Jan 2005 at 10:14