User interfaces in Iron Man

First off: possible spoilers ahead!

Iron Man was an excellent movie. However, since this blog is not dedicated the movie reviews, I thought I’d discuss some of the user interface elements involved instead.

This video highlights two UI elements that were well thought-out. There are more I’d like to discuss, but I couldn’t find them. There are two clips: one is of a holographic prototyping interface and the other is the Iron Man suit user interface and flight interface:
Click to download (6.5 MB MP4 video)

Holographic Prototyping and Direct Manipulation on the Cheap:

The first clip ends at around 26 seconds and shows off Tony Stark’s (aka Iron Man) holographic prototype interface. The hologram is a sci-fi cliche, but its usefulness is immediately evident. Direct manuipulation has been discussed before, and this typically requires something physical to manipulate. These physical prototypes are expensive to fabricate, especially if multiple revisions are needed. In the clip, Stark has already built out the specs for this piece of his suit, and he’s able to add and subtract parts and accurately visualize the effects of the modifications without fabricating lots of physical prototypes. The coup de grace is when he is able to stick his arm inside of the hologram and test it out. It’s direct manipulation of a prototype without the expense. Because this kind of manipulation is so natural, very little cognitive effort is needed to use this interface.

Flight Suit Interface and Transfer of Training:

The second clip starts at 27 seconds and demonstrates part of the suit’s user interface and the flight interface as well. The flight interface is very similar to that of a fighter jet, which will make transfer-of-training easy from a fighter jet to an Iron Man suit. This will cut down on the need to train users of the Iron Man suit - if they can fly a fighter jet, they can fly this suit.

Main Suit User Interface Voice Commands:

What was most interesting in the clip was how the general UI was controlled. The heads-up display is directly in front of the user’s face, but the user cannot touch the display. Therefore, direct manipulation is out of the question. The suit does take voice commands, as shown in the video. This is an obvious choice, but it is slow to use. Imagine flying at some insane speed under high stress - do you want to have to yell out a command that takes several seconds to issue, then wait for a reply from the suit? Probably not a good idea. The closest thing to voice interaction in this world is the Microsoft Sync system. This system integrates bluetooth phones and MP3 players into Ford cars and is all voice controlled. When it works, the eyes stay on the road for longer and less attention is required to make a phone call or play music. But when it doesn’t work, the error correction is simply a mess. It takes a huge amount of effort and is bad for driving or flying.

This is a great review of the Sync system (see 1:45 for an example of a Sync error as how the user simply gives up):

Main Suit User Interface and Eyetracking:

Besides voice commands, the other control option is eyetracking. If the eyes are focused on something in the environment, a command can be issued to zoom in, take a picture, etc. Issuing that command, however, would have to be done using a voice command or a button-press. The eyes are needed to focus, so something other than the eyes must issue a command. This is not the ideal situation because it requires coordination of multiple systems - the eyes must remain focused on the target while another body part confirms the command. Overall, though, this is not too much of a problem. It is similar to tracking the cursor on a computer screen and clicking the mouse with your finger. Nonetheless, it is a less elegant solution, especially during flight or in combat and the hands are required for another task.