Characterizing user performance in command-driven dialogue

To learn to use an interactive system, a person typically has to acquire a good deal of new knowledge. The ease of learning will depend on the extent to which the design of the task and the interface capitalizes on the user's pre-existing knowledge and his or her cognitive capabilities for learning. This paper explores the nature of both design decisions and user learning with a command-based system. Three studies were conducted, all involving a task in which secret messages were decoded by means of a sequence of commands (based on the task used by Barnard et al. In Study I, software specialists designed command structures for the task and gave reasons for their choices. In Study II, naive subjects chose between alternative command terms. In Study TTI, subjects learned to use interactive versions of the task in which dialogue factors (command terms and argument structures) were systematically varied. The results enabled the development of user knowledge of the system to be specified in detail. Comparisons across the three studies highlighted the diversity of the factors determining both design decisions and user behaviour.