Archive for category user testing
Book Review: Working Minds
Posted by jasonwong in decision making, good design, user testing on August 31st, 2009
Working Minds: A Practitioner’s Guide to Cognitive Task Analysis is a book designed to introduce the domain of Cognitive Task Analysis (CTA) to those who have an interest in learning how to document cognitive expertise. In today’s information-heavy, cognitive-dependent world, determining how experts think (specifically, how they make decisions) is important to capture when training novices or designing systems that will aid that expert in his or her work.

Beth Crandall, Gary Klein, and Robert Hoffman all have a great deal of expertise in the field, and the book certainly represents that wealth of knowledge. The book is structured into three broad sections: an overview of CTA methods, a section the authors call “Finding Cognition”, and a section on specific applications of CTA. This is all followed by an appendix entitled “Guidance for Data Collection.”
I found the book organized in a confusing manner. The “Finding Cognition” section of the book laid out some of the problems that CTA is designed to solve, such as the rise of new information technologies. Additionally, the authors discuss their theory of macrocognition, which takes a broader, real-world view of cognition (versus the localized, specific cognitive phenomenon studied in the laboratory). I found this useful and wish the book started off with these chapters.
Cognitive Task Analysis relies on expert interviews in order to document aspects of macrocognition, and the interviews are often difficult to conduct. I kept wondering how to conduct an effective interview only to find those tips and tricks stuffed into the appendix. I would have rather had the appendix made an actual chapter and tied in with the first section that gave an overview of CTA methodology.
A downside I felt regarding the author’s expertise is that they have completed many successful CTA projects using their specific techniques, and those are the primary focus here. The book is not a survey of CTA methods, but instead a detailed description of a subset of methods that have worked well for the authors. This is disappointing, but understandable in that it allows for the authors to explore a few techniques deeply and ensures that the authors speak from experience.
I thoroughly enjoyed the final section of the book, which detailed applications of CTA from training, systems engineering, market research, and more. A convincing case is made for the usefulness of CTA when dealing with decision making in any context, and the rest of the book is designed to give the reader the tools with which to understand how decisions are made.
One final note: the authors often discuss conducting interviews that can take place for several hours or over the course of several days. This is realistic for these researchers, who run a company that is consulted by other agencies when they really need a CTA completed for a project. However, for someone like myself, there is no way on Earth that I am going to be able to conduct two-hour long interview sessions with any sailor. Therefore, the tools and methods presented in the book would have to be adapted for anyone working in a realistic environment. Despite this, the book provides an excellent starting point for learning about CTA and conducting CTA studies.
Ubuntu Install Instructions: Confusing and Non-Working
Posted by jasonwong in user testing on April 23rd, 2009
Ubuntu is a Linux distribution that is fairly popular these days. It mostly lies in the realm of computer enthusiasts, but more and more regular people are trying it out. I recently got a Dell Inspiron mini 9 netbook that I loaded with Windows XP. However, Ubuntu just released a new version of Ubuntu today, so I figured I would try it.
The newest version has a “Netbook remix” edition, which is streamlined for these underpowered little machines that often don’t come with a CD-ROM drive. Mine doesn’t, but Ubuntu has instructions for how to put the image on a USB flash drive. My netbook can boot from that drive and install Ubuntu that way. I figured, why not. Here are the instructions for OS X, which is my main operating system (website here):

I am pretty comfortable with the Terminal, so I didn’t think it would be a big deal. I even managed to get up to Step 5 without getting confused. Then comes Step 6.
From Step 5, I determined that the device node for my flash drive was, in fact, /dev/disk2. So Step 6 says to Run diskutil unmountDisk /dev/device/node. Two issues. Firstly, the command is listed in monospace type, and this convention typically means you should type the command in exactly as you read it. However, you are SUPPOSED to replace /dev/device/node with the device node name you got from Step 5.
Secondly, the device node name from Step 5 was /dev/disk2. The instructions show /dev/device/node – three slashes instead of my two. Was I missing something? Did I need /dev/disk2/something else? Apparently not. I WAS supposed to replace /dev/device/node with /dev/disk2. That does not match in terms of the number of items between slashes, so knowing what to replace with what was not apparent.
So, on to Step 7. Execute sudo dd if=/path/to/downloaded.img of=/dev/device/node bs=1M. Again, there is the problem that /path/to/downloaded.img needs to be replaced by the actual path of your disk image. But once I passed that hurdle, I got the following error message:
dd: bs: illegal numeric value
That, of course, is a technical problem, and not a human factors problem. What IS a human factors problem, though? A simple set of 8 instructions that are confusing, unclear, and don’t even work!
That bodes poorly for actually using Ubuntu, doesn’t it? I think it does.
Woah – user testing gone too far?
Posted by jasonwong in user testing on March 20th, 2009
Douglas Bowan, a designer at Google, is leaving the company. His blog post said that he “introduced Visual Design as a discipline to Google,” which sounds like an impressive feat. But why is he leaving?
Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such miniscule design decisions. There are more exciting design problems in this world to tackle.
Woah. Sometimes, good design trumps usability, and that’s bad. Most human factors professional would also emphasize the importance of user testing. But testing 41 shades of blue? Requiring data on whether a 5-pixel border is too wide? Yeah, it’s time to move on. Fascinating stuff, though.
XBox 360: New and less-cluttered Dashboard
Posted by jasonwong in clutter, data visualization, user testing on July 15th, 2008
Only two weeks ago, I posted about an excellent analysis of the XBox 360 Dashboard and a potential redesign. It would have been a great new interface. Microsoft, however, had other ideas.
Yesterday at the E3 conference, Microsoft released details of their Dashboard update this Fall. Frankly, it looks far more modern and far less cluttered. The “blades” have been replaced with a Zune-like menu that should be easier to understand. From a human factors perspective, the biggest feature is: less clutter! The interface is much cleaner with an emphasis on the important UI elements.
A big deal was made on making it easier to find downloadable content. I haven’t seen any screen shots showing an improved search feature, but anything would be better than the current system. Hopefully there will be screenshots to come!
Incredible analysis of a display
Posted by jasonwong in clutter, data visualization, good design, user testing on July 2nd, 2008
The XBox 360 dashboard is the user interface to the XBox operating system. When the game console is booted up, the user is presented with a display that looks like:
The user can navigate to different sections of the display using the XBox 360 controller to switch “Blades” – between, Games, Media, and other functions. It’s a fairly usable design, though it is difficult to make sense of initially (at least, based on my experience).
Over at the blog The Fanboys, there is a fantastic analysis of the 360 dashboard display. The dashboard is broken down into pixels and classified as being used for the user’s content, interactive items (buttons, menus, etc.), ad space, or blank space. The results are startling but also inform a smart redesign that minimizes dead space but does not lead to increased clutter. It’s a really impressive redesign.
The Fanboys: Dreaming of Dashboard 2.0
As someone who is starting to propose a display redesign for a submarine tactical system, this kind of analysis could be incredibly useful to implement. At the very least, it gets the mind thinking in a visual, yet quantitative, manner. Oftentimes, it is easy to be descriptive about changes that need to be done. But when you get sensible and realistic numbers, the case becomes far more convincing.

