Designing Interfaces that Stimulate Ideational Super-fluency

In recent research, we have demonstrated that the presence, basic features, and match of an interface to a task domain can either stimulate or impede students’ ideational fluency during science problem-solving activities. The first goal of our research was to investigate whether the presence of computer interfaces, which are primarily perceived as interactive communications tools, have affordances that elicit greater total communicative fluency than hardcopy pencil and paper tools that tend to be associated with non-interactive note-taking. The second goal was to explore whether interfaces characterized by different input capabilities, such as keyboard versus pen, have affordances that prime qualitatively different communicative actions. For example, will pen interfaces selectively stimulate increased communicative fluency in nonlinguistic representational systems (i.e., numeric, symbolic, and diagrammatic content), while in contrast keyboard-based graphical interfaces prime increased communicative fluency in linguistic representations? A third goal was to assess whether increased communicative action involving representations that are well matched with a task domain (e.g., diagramming for geometry problems), which effectively increases students’germane load or effort compatible with the task, also facilitates a parallel increase in their appropriate ideation and problem solution correctness (Sweller, Van Merrienboer & Paas, 1998). In contrast, interfaces eliciting increased communicative actions that are poorly matched with a task domain actually may serve to undermine or impede performance within the domain. A fourth objective of our research was to document the relation between active forms of nonlinguistic marking that students make to structure information while working on problems (i.e., diagramming, informal “thinking marks” placed on problem visuals) and the correctness of their related problem solutions.

As theoretical background, according to Affordance Theory, people have perceptually-based expectations about objects in the world, including constraints on successful performance that differentiate one from another. These affordances establish behavioral attunements that transparently but powerfully prime the likelihood that people will act on them in specific ways (Gibson, 1977). Since computer interfaces are associated with communications, interface affordances may be expected to elicit a general increase in communicative acts from people while using them to complete tasks. Furthermore, people’s expectations about constraints on the type of communicative acts supported by different interfaces may influence the content that people actually communicate when using them. Activity Theory, which in many ways is complementary to Affordance Theory, maintains that communicative activity plays a major role in mediating, guiding, and refining mental activities, which is manifest in hypotheses, problem solutions, and other ideational phenomena that people generate (Luria, 1961; Vygotsky, 1962). It has been documented that as tasks become more difficult, people do indeed spontaneously increase communicative actions such as self-talk, gesturing, and written marking, and these communicative actions also are effective strategies in improving performance (Berk, 1994; Comblain,1994; Luria, 1961; Vygotsky, 1962).

The implications of these theoretical views for interface design, especially for enhancing performance in areas such as education, are to strive to support:

• user input that is active, rather than passive
• richly expressive user input in terms of breadth of representational systems covered (i.e., linguistic, numeric, symbolic, diagrammatic)

A new generation of rich communications interfaces will need to be designed that are capable of stimulating active communication during problem-solving activities. To support difficult or extended problem-solving tasks, especially in domains such as STEM (science, technology, engineering, mathematics), interfaces also need to be designed for broad coverage and flexible shifting among different representational systems (Oviatt, Arthur & Cohen, 2006). For example, it would be common while solving STEM problems to begin by diagramming, and then work out a solution that involves expressing numbers and symbols, followed by summarizing an answer in linguistic terms. Pen interfaces can support all four of these representational systems, including expressing nonlinguistic forms of representation (e.g., diagrams, symbols) that are not well supported by traditional keyboard-and-mouse interfaces.

To investigate these issues, a longitudinal study was conducted on biology students’ ability to generate appropriate hypotheses and also solve problems while using different hardcopy and computer interface tools, including: (1) non-digital paper and pencil materials (i.e., existing work practice), (2) a digital pen and paper interface (Anoto, 2009), (3) a pen tablet interface, and (4) a graphical tablet interface incorporating a keyboard, mouse and pen. Within-subject comparisons were performed of how well the same students completing the same problems performed on hypothesis generation and problem solving tasks, simply as a function of using different interface tools. To ensure that interfaces are designed for diverse users, eight participants in the study were high-performing students and eight were low performing ones. Planned comparisons focused on measures of communicative fluency, ideational fluency (i.e., number of appropriate biology hypotheses generated), and correct solutions on problem-solving tasks.

As predicted, students’ communicative fluency was heightened when using computer interfaces, compared with hardcopy paper and pencil. Also as predicted, the two types of pen interface primed significantly higher levels of nonlinguistic communicative fluency, whereas the keyboard-based interface primed higher levels of linguistic fluency. In parallel, the pen interfaces facilitated higher levels of scientific hypothesis generation, compared with the keyboard-based interface and hardcopy pencil and paper tools. Finally, higher rates of active pen marking (i.e., diagramming, informal “thinking marks”) were observed to be associated with substantially higher solution correctness on problem-solving tasks. In our presentation, we will review the specifics of these data and conclusions.

In conclusion, our research reveals that computer interfaces have affordances that can increase communicative fluency, and also substantially facilitate ideation and problem solving. One important theme in this research is the role that an interface can potentially play as a facilitator of people’s own communicative activity, which in turn can prime related mental activity if well matched with a task domain. This research also highlights the importance of designing for a “digital literacy” that is far more active than present interface conceptualizations, and that encompasses both linguistic and nonlinguistic forms of representation (Larkin & Simon, 1987; Oviatt, Arthur, Brock & Cohen, 2007; Schwartz & Heiser, 2005). Pen interfaces, or potentially multimodal ones that incorporate them, provide a single focused input tool for fluently expressing varied representational systems, including nonlinguistic ones that are critical for domains like STEM and for real-world problem-solving activities. In the future, digital interfaces need to be designed as richly expressive communications tools, ideally with the ability to accommodate multiple representation systems, multiple modalities, and multiple linguistic codes.

Works Cited

Anoto . Web. <http://www.anoto.com/>.

Berk, L. E . “Why children talk to themselves.” Scientific American 271.5 (1994): 78-83. Print.

Comblain, A . “Working memory in Down’s Syndrome: Training the rehearsal strategy. ” Down’s Syndrome: Research and Practice 2.3 (1994): 123-26. Print.

Gibson, J . “The theory of affordances.” Perceiving, Acting and Knowing, Ed. R. Shaw, J. Bransford. Hillsdale, N.J.: Erlbaum , 1977. 67-82. Print.

Larkin, J. and Simon, H . “Why a diagram is (sometimes) worth ten thousand words.” Cognitive Science 11 (1987): 65-99. Print.

Luria, A . The role of speech in the regulation of normal and abnormal behavior. Liveright, 1961. Print.

Oviatt, S. L., Arthur, A. & Cohen, J . “Quiet interfaces that help students think.” Proceedings of the Nineteenth Annual ACM Symposium on User Interface Software Technology (UIST’06), CHI Letters. New York: ACM, 2006. 191-200. Print.

––––, and Brock, Y . “Expressive pen-based interfaces for math education.” Proceedings of the Conference on Computer Supported Collaborative Learning: Of Mice, Minds & Society, International Society of the Learning Sciences . Ed. C. Chinn, G. Erkens and S. Puntambekar. 2007. Print.

Schwartz, D. and Heiser, J . “Spatial representations and imagery in learning.” The Cambridge Handbook of the Learning Sciences. Ed. R. K. Sawyer. Cambridge: Cambridge UP, 2005. 283-98. Print.

Sweller, J., Van Merrienboer, J., Paas, F . “Cognitive architecture and instructional design.” Educational Psychology Review 10 (1998): 251-57. Print.

Vygotsky, L . Thought and Language. Trans. E. Hanfmann and G. Vakar. Cambridge: MIT, 1962. Print.