Keystroke-Level Modeling (KLM)

To analyize the effect of our tool selection design changes on efficiency, we used CogTool (www.cogtool.com) to create a Keystroke Level Model that predicts tool selection times.

The model assumes the following:

To select a frequently used tool in the new design, a pie menu user would:

  1. Right-click and hold on the image viewer
  2. Move the cursor to a pie menu slice
  3. Release the mouse button.

For an infrequently used tool in the new design, a user would:

  1. Right-click and hold on the image viewer
  2. Move the cursor to the "Show All" pie slice
  3. Release the mouse button
  4. Move the cursor to the desired tool
  5. Click the mouse button

For any tool in the old design, a user would:

  1. Right-click on the image viewer
  2. Move the mouse to the desired tool
  3. Click the mouse button

Here are the results of our model:

 

Original Design

Redesign

Selection of frequent tool

2.90s

2.68s

Selection of frequent tool

2.90s

3.27s

Our model predicted an average tool selection time for commonly used tools of 2.68 seconds for the pie menu design, versus 2.90 seconds for the existing right-click menu. That represents a speedup of 7.5%. While this may sound like a small improvement, radiologists will perform tool selection operations hundreds or even thousands of times per day. For less frequently used tools, the model predicted that our design would result in a tool selection time of 3.27 seconds, representing a 12.7% slowdown. On balance, we believe that radiologists will select the most frequent tools far more often than the less frequent ones, so this tradeoff is well worth it. This does, of course, depend on the relative frequency of use of "frequent" vs "infrequent" tools. We do not have empirical data to support our assertion that the most commonly used tools are used far more frequently than the others. However, our observations suggested that this was the case.

It is also worth noting that KLM does not account for Hick's law, which states that the the time required to make a decision increases logarithmically with the number of available options. Applied to menu design, selection time on a set of undifferentiated buttons increases with the number of buttons. Categorization of buttons negates the effect. We therefore believe that because of Hick's Law's impact, the difference in selection time for frequently used tools will increase, further improving selection time in our design, and the difference in selection time for infrequently used tools will decrease, minimizing the cost of the extra clicks required to access these tools.