CitiStat User Test I

Description | Test Set-up | Findings | Links to Data

Description

General

This meeting with our actual users was part Contextual Inquiry and part Think-Aloud user testing session.

Due to the confusion with the selection order of the graphing options in the previous user test, we wanted to find out if our actual users had a specific way of creating graphs. Our design discussions also deliberated over whether to allow graphing metrics in both the x-axis and y-axis. So the Contextual Inquiry will be structured to get these specific users' mental model and graphing needs.

In our consolidated VB prototype, we came up with two different ways to select more than one metric in the tree hierarchy, namely using the control-click, and using checkboxes next to the tree items. Having our real users to use these two methods could help us determine the more intuitive one more confidently. Since this was the first Visual Basic (VB) prototype that we had developed, this first user testing session with the real users was a good opportunity to assess their initial reaction to the various components of the interface.

Test Set-up

Goals

• Find out if they really need to create graphs with metrics on both the x and y-axis
• Why do they do certain graphs, e.g. graph with both lines and bars?
• Find out what goes into their process of creating a graph
• User-test individual components of the system:
• Expand / Collapse All functions
• Metric search function
• Navigating using the tree hierarchy

Findings

Graph Types

• They chose the graph type based on subjective preferences, namely what looked better, whether or not comparison of two or more metric was needed. There is a need to be able to switch between graph types easily.
• Graphs with metric on both axis are not common and not used AT ALL. Users could not conceive the use of such a graph even when shown one.

Graphing Process

• In Excel, all the users selected the chart type first, then selected the data range, followed by the options. This order of selection was also seen when they used our prototype.
• They directly edited the graphs in Excel at times. There is a need to be able to edit labels in the graph directly.

Expand / Collapse

• Only one user noticed this function, but didn't think it was useful.

Search

• All the users overlooked this function. But they all expressed interest after we showed them where they may doing searches of metric based on keywords.

Tree Navigation

• All the users preferred use of checkboxes to multi-selection with "Control" and "Shift".
• All of them selected the wrong metric for scenario 1. There is a need for better labeling of metrics.

Links to Data

Prototypes