User Test I

Description | Test Set-up | Findings | Links to Data

Description

General

After exploring five radically different design concepts, we came to a point where we needed empirical data to justify consolidation decisions. The five interaction styles of each of the design concepts have their strengths and weaknesses, and we needed to know if they could be supported by user data. We also wanted to find out from this test which design was most superior in terms of navigating through the metric, building graphs and being the most intuitive.

Due to the nature of paper prototypes, it is not easy to test click-and-drag interactions, as well as the affordances of some widgets (whether a button looks clickable). But they are sufficient to investigate the users' reaction to the interfaces and general graphing mental models.

Test Set-up

Goals

• Which interaction style is more intuitive for creating graphs?
• Can the users understand the interface?
• Can the users navigate to a required metric easily?

Scenarios

The scenarios are deliberately set up simple enough for the low-fidelity prototypes. Users are asked to graph a particular metric in a given time for some divisions. For the prototypes which allow users to graph metrics in both the x-axis and y-axis (i.e. Mac and Pivot), the users are asked to do so.

Findings

Overall

• Graph labeling is necessary, e.g. unit of measurements, x-axis and y-axis labels.
• Some form of instructions have to be given to guide users on how to begin.
• A preview of the graph to be created or the graph type provides very good feedback to the users.
• Some form of instructions have to be given to guide users on how to begin.
• Some categories and metrics are similar enough to cause confusion. It is important to help distinguish between similar metrics.

Excel+

• Buttons to expand design failed miserably and the user did not understand how to drag information over.
• There is confusion between what will appear on x-axis and what will on y-axis.

Mac

• Users are not familiar with the 'Mac' interaction method of navigating through the metric hierarchy using the column view.
• It is frustrating for users to have to go through the entire process of selecting a metric each time they want to add a new line to a graph.

Pivot

• Users have problems understanding the graph. It needs to be more properly labeled.
• Users could not figure out how to select the metric and create a graph. It is not very intuitive that those metric labels can be clicked and dragged.

Tabs

• There is confusion to what order to select the graphing options.
• Users did not know that those numbers on the tabs represent divisions instead of step numbers in a process.

Tree

• The user avoided the tree and clicked elsewhere before attempting to look for metrics in the tree.
• Users expected a "Create Graph" button after selecting a metric. There is not enough feedback when graph is automatically created.

Links to Data

Prototypes