From CRUD to CREAM: Imagining a Rich Scholarly Repository Interface

Overview

This article attempts to think through the extension of the prototype of a dynamic repository interface designed to support a rich range of scholarly activities associated with text. The Just-in-Time-Research or JiTR prototype was developed by a team led by Geoffrey Rockwell, with the idea of “mashing texts” together to allow for an agile Web environment to support the gathering and maintenance of a set of texts for scholarly research (Ruecker, Rockwell, Brown, Sinclair, & Organisciak, 2009). Built into the user scenarios, wireframes, and prototyping for the JiTR project was the idea not merely of gathering a set of textual materials for research purposes but also of being able to apply metadata to the objects, to clean or edit them, and to prepare them for text analysis tools. The initial prototype of JiTR incorporated all of these activities. The prototype was built on a relational database with the four fundamental functions associated with persistent storage: create, read, update, and delete, summed up by the charming acronym CRUD. Starting from a sense of increasing importance of the large collection interfaces (Rockwell, Ruecker, & Ilovan, 2011), this article explores the strategy of rejecting, for the purposes of interface design, these unappetizing primitives for the richer yet still basic functions required by the interface for a dynamic collaborative scholarly repository. This is the more appealing kind of environment that the Canadian Writing Research Collaboratory (CWRC) seeks to design and implement for scholars of writing in and about Canada.

The goal of the CWRC project is to support a wide range of online research activities associated with the study of writing, ranging from primary text editing projects to born-digital scholarship to group-sourced projects with considerable multimedia content. It aims to support collaboration in the processes of object creation, enhancement, and re-use, and, as part of the suite of functionalities required for these processes, is to support the iterative creation and analysis of derivative collections and objects. In other words, although the basic actions of the database summarized by CRUD may remain the same, the CWRC will need to support an extensive array of object- and collection-level metadata that will allow a diverse array of digital objects to be manipulated and recombined by different users for various purposes, all within an environment that allows for those objects to be changed by users with appropriate permissions at any given time. The interface required to support such a complex set of affordances constitutes a major challenge, one that we are exploring through an extension of the idea of rich prospect, which has been developed in relation to browsing, to that of interface functionality. It is noteworthy in this context that the Web development community circulates various software libraries as means of achieving “rich” Web interfaces, without reflection on what is meant by richness as a descriptive category, and that at the same time there is a strong impetus in Web design theory towards minimalism (Carroll, 1998; Chapman, 2010; Hackos, 1999). Indeed, the hugely influential heuristics developed by usability engineer Jakob Nielsen conflate “Aesthetic and minimalist design” (Nielsen, 1994, p. 30; Nielsen 2005). There is a conceptual tension between the idea of richness and that of minimalism, and the question of whether this translates into an actual tension in interface design bears consideration.

In rich-prospect browsing, we have the idea of prospect as provided by an overview of the material that consists of a meaningful representation of all the parts, and of richness in the form of tools that the user can apply in manipulating the prospect display. The argument is that the number and sophistication of the tools should emerge from the variety and complexity of the metadata available, whether that metadata has been manually produced, as in XML encoding by human readers, or algorithmically produced, as for instance through text mining results. This article applies that notion of richness to the wide range of activities required by a scholarly work environment, and considers how interface design based on the principles of “rich prospect” can be combined with the idea of functional primitives to organize an interface (Ruecker, Radzikowska, Sinclair, 2011; Ruecker, 2006). It places that contention in dialogue with the results of a user-based design exercise conducted to begin to probe how scholars themselves might perceive the organization of a rich array of functionalities.

The notion of primitives suggests itself as a means of offering prospect while not overwhelming users with the entire range of possible actions related to a single primitive. John Unsworth’s widely embraced taxonomy of scholarly primitives suggests that our activities can be understood in terms of discovery, annotation, comparison, reference, sampling, illustration, and representation (Unsworth, 2000). Unsworth’s list of primitives, while attempting to distil the “the irreducible currency of scholarship” (Unsworth, 2000, para 7) across all eras and media, was constructed with scholarly activity based on networked digital information in mind. A dynamic scholarly repository clearly needs to support all of these. In addition, it needs also to consider some additional functionalities, which, if not scholarly primitives per se, are essential to the actual management of research materials in an online environment.

The proliferation of functionality in an interface makes for a challenge, however. User studies of rich interfaces such as those of the Networked Infrastructure for Nineteenth-Century Electronic Scholarship and the Orlando Project have revealed that despite the creators’ desire to present the rich array of functionality to users up-front in their initial instantiation, that very array of affordances can be bewildering and intimidating, suggesting that a simpler interface is more effective despite the desire to create functionality that pushes users towards more sophisticated use of a resource (Nowviskie, 2007; Wheeles, 2010; Brown, Menezes, Nonnecke, Ruecker, & Warwick, 2009; Brown, Clements, & Grundy, 2006; Brown, Clements, Elio, & Grundy, 2006).

The functionalities associated with this scholarly environment designed to help users separate and make use of features of value can be summarized under the rubric of CREAM:

Create — represent, illustrate

Read — sample, read, compare

Enhance — refer, annotate, but also manipulate e.g., extract entities, apply markup or metadata, strip superfluous tags

Analyze — discover/search, select, visualize, mine, cluster

Manage — track, label, process, transform

We thus offer CREAM as a rubric under which a more manageable number of categories of functional primitives can be grouped. This might help to organize the JiTR interface conceptually as we develop it towards supporting a wide range of scholarly activities online.

Extending JiTR

This article will imagine the further extension of JiTR, particularly the structured surfaces prototypes designed by Milena Radzikowska as a member of the INKE project to support information organization by providing a “cognitive interface artefact that provides a layer of meaning that supports the data imposed upon it” (Radzikowska, Ruecker, Brown, Organisciak, & the INKE Research Group, 2011, pp. 3–4). Structured surfaces are an attempt to generalize the functionality that has become widespread in the use of digital maps with interactive pins. The maps provide a layer of information (e.g., geography, streets with their names, and named locations), but that layer is augmented or extended when the user adds pins that represent a new category of information (e.g., restaurants; birth places of authors). In structured surfaces, we have replaced the maps with a variety of data visualizations.

Figure 1. This sketch shows a rose diagram as the underlying surface, with informational pins appearing as yellow dots.

We imagine JiTR developed in the complementary directions of a rich prospect browser capable of providing a rich overview of collections of thousands of objects and a reader or workflow manager capable of allowing for a wide range of interactions with collection objects at the level of the individual item.

In order to evaluate the rubric and our provisional set of primitives, we tested them using a card-sorting activity of the kind typically used by website designers in user studies to understand the categories under which people naturally group the functionality, or more often the content, of a particular site. This study was conducted in September 2011 with CWRC scholars, as a step towards designing the interface.

Card-sorting methodology

The card-sorting was conducted at a CWRC workshop by Susan Brown, Susan Hesemeier, Ashley Moroz, Mariana Paredes, and Megan Sellmer. Having no prior experience of card-sorting exercises, the team was guided by the helpful protocol outlined by Donna Spencer and Ted Warfel (Spencer & Warfel, 2004).

The workshop, held in late September 2011, was attended by researchers already affiliated with or interested in CWRC, although the former constituted the vast majority of the participants. The 17 participants were divided into four groups of between three to five researchers. Most participants came from a humanities discipline such as English (the majority), French, or History, with a couple from library or information science; they ranged in age from early twenties to late seventies, and from Master’s students through to retired academics.

The extent of knowledge about CWRC functionalities held by participants varied considerably; indeed, one positive side-effect of the card-sorting exercise is that quite a number of participants seemed to come away with a fuller sense of what CWRC aims to provide. The earlier parts of the workshop made them familiar with some of the activities, since they had been exposed to and were asked to provide feedback on a range of analytical tools and visualization prototypes, an editor and metadata editor prototype, an experimental reading interface, and also a couple of interfaces for workflow management. They were also aware of the fact that we plan to call the CWRC repository itself ORCA, short for Online Research Canada and a more memorable and pronounceable acronym than CWRC.

Each group was given a set of 94 cards indicating functionalities, indicated by roman numerals and ordered alphabetically, beginning with “I. Accept/reject a correction by others of materials in your collection”; “II. Add a new item to ORCA”; and “III. Add a person, organization, text name, or placename to the CWRC indices” and ending with “XCIV. Write an essay, article, author entry, or topic entry.”

In addition to these 94 cards, they were also given a set of different coloured cards with suggested categories as follows:

• Analyze
• Browse
• Collaborate
• Create
• Enhance
• Explore
• Home Page
• Manage
• Map
• Net/work
• Persistent Toolbar
• Read
• Visualize

Note that while the CREAM categories were present, they were not the sole categories provided. Some categories were taken from an interface sketch presented earlier in the workshop. The groups were also provided with blank cards for both the activities and the categories so they could create their own categories, and were told they could discard or modify any of the cards as they saw fit. They were told that they could establish subcategories as well.

Each group had embedded in it a member of the central CWRC team to help facilitate the exercise and clarify any questions about functionalities as they arose, but the facilitators did not participate in the sorting decisions themselves. The groups worked diligently at sorting and took varying amounts of time to complete the task. One group had to be prodded considerably in order to finish in about an hour, and if this activity were repeated we would allow more time (we were forced to hurry it due to time constraints). By the end, however, each of the groups had managed to deal with all their functionality cards, either by sorting them into a category or discarding them. There was considerable modification and invention of categories, and considerable discarding (as redundant), annotation, or modification (but not invention) of functionality cards.

Figure 2. Results of one of the card-sorting groups, showing considerable revision and invention of both categories and, to a lesser extent, functionality descriptions

Analysis

Analyzing the data proved to be a challenge, and at first glance it seemed that the groups’ choices were so diverse that we would get little guidance from the exercise. However, the cards for each group were organized into a spreadsheet, and then the results for each card were summarized, revealing more agreement than was at first apparent. (See the Appendix for a tabular summary of the results.) There was complete disagreement in the case of 20 cards, or 21%; agreement between two groups in 37 cases or 39% of cases; between three groups in 18 cases or 19%; and unanimity in 13 cases or 14%. A further 6 cards or 6% were discarded or not categorized by two or more groups. The spreadsheet of the results was analyzed and then, because the categories and data format were not well suited to computational analysis, they were manually clustered into groups.

There was a high level of agreement amongst users on some aspects of Create, Manage, and Search. For instance, all four groups agreed that the following activities belonged in the group “Create” or in a similar category (“Build Project” or “Create/Input/Contribute”): adding a new item to the repository, creating a new document, inputting a draft of a text, starting a project or a project profile page, writing an essay, article, or entry, or uploading a scan of a primary text. Three out of the four groups also agreed that something like “Create” was the best place to categorize adding a person, organization textname or placename to indices, adding a bibliographical item, or uploading materials from the user’s computer.

“Search” was pretty close to “Create” in terms of the number of unanimous placements. Here too there was innovation on the category name by way of expansion to “Search/Browse” for one group and “Search/Browse/Read/Explore” for another. Free-text searches, refinement of searches by faceting, saving a search, and sorting a set of search results manually all seemed to belong here. Three of the four groups also, rather interestingly, grouped not only advanced searching but also forming a timeline from search results and mapping a set of results as belonging to “Search.” This might well not have been the case if the functionality had been described as “forming a timeline from a collection” or “mapping a collection,” which drives home the potential of particular choices of terminology to sway the results.

One finding that is quite interesting is that Search is a less clear-cut category than one might have thought. Forming a timeline based on search results, or viewing a set of search results on a map, for example, were both considered to belong to this category by three or more groups, as was sorting through search results, along with search activities themselves. In addition, Search and Browse, which are often separated within interfaces, seem to blur into one another quite a bit, so that we ended up with hybrid “Search/Browse” and “Search & Read” categories that included things like sorting search results as well as listening to audio or viewing images or videos. Yet there was not a great deal of agreement amongst the groups as to what fell within those categories, with at most two of the groups lining up.

“Manage” was the third category that contained a fairly high number of functionalities; it lagged slightly behind “Search” in number of items and extent of agreement. We suspect that this user population may not be accustomed to thinking of managing their own online collections. One group joined it to “Admin Tools” with a slash; others nested it in a hierarchy to clarify it, as in “Manage > Manage Site (administration).” All four groups agreed on this as the right place for approving another’s annotation, defining a workflow, and signing off on a workflow stage. Three groups found checking workflow status prior to publication, checking a set of materials prior to publication, rolling back a change in a document, and checking storage quota also to belong in this group.

The “Analyze” category was considerably undersubscribed: even more than search, analysis apparently bleeds into reading and other activities for many scholars. It is also worth noting, however, that some participants probably had their first exposure to real text analysis tools only the day before the card-sorting exercise occurred and therefore were unlikely to consider this a major component of their online research activity. It is therefore worth considering how best to incorporate such tools within a rich scholarly interface so as to promote their use.

Collaborate, on the other hand, had four items agreed upon by three groups: communicating with another researcher within CWRC, finding who else is working in a related area, and sharing work in progress. Accepting or rejecting a correction by others of materials in one’s own collection, communicating with the CWRC team, and establishing a personal profile also belonged here according to two groups.

“Enhance” saw little agreement, being applied to such diverse activities as labelling named entities in a text, supplying bibliographical information, or commenting on an image, audio, or video file, and vying with such categories as “Create,” “Collaborate,” “Read/Analyze,” and user-added categories, such as “Work” or the more common “Edit.” One group used it fairly extensively, another made it a second-tier category, and the other two did little or nothing with it. Yet, despite the lack of consensus around this term, we would argue that it makes sense to hang onto it, at least pending further study. For one thing, while it may well be that in the long run another term such as “Contribute” turns out to be better, none of the categories proposed by these groups show a greater degree of consensus. For another, it is important to remember that while members of this community have embraced the prospect of working collaboratively within an environment that allows for ongoing enrichment — whether by means of correction, expansion, application of markup or other metadata, or other activities — of shared collections of scholarly materials, few have any experience of such functionality, and are therefore likely to fall back on modes of improving existing work with which they are most familiar, such as editing. In this case, then, we chose to retain a category that was not entirely successful, both because the exercise did not reveal a better term and because of a sense that in this particular case, our respondents’ input needed to be weighed against the development team’s understanding of the system. Despite the fact that in this case our judgment was contrary to that of many of our respondents, we take seriously the challenge identified by their defamiliarized view of things: it at best identifies a problem with vocabulary but more likely points to a major impediment to communicating this conceptual grouping to our community of users.

Card-sorting results

As indicated, the analysis points to some revision of the initial categories as they were distributed to the groups. In addition, functional definitions of the categories, which had been given to the participants undefined, emerged from the ways in which particular items clustered. The results are summarized in Figure 3.

Figure 3. Breakdown and definition of categories resulting from card-sorting

As the “Undetermined” group indicates, a good 28% of the functionalities didn’t cluster at all, because each group saw them as belonging to different categories, suggesting that they were very hard to place conceptually within the rubric provided. Among these were various kinds of annotation and administrative functionalities associated with the repository itself, such as membership management, announcements to CWRC members, privacy settings, password management, download/backup, and export of sets of materials. However, scraping a website to form a collection, sending results to a CWRC tool, visualizing a text or collection, creating links, correcting errors, and viewing a version history also completely divided the groups. These results, along with the overall diversity of the respondent groups’ sorting strategies, drive home Jennifer L. Bowie’s contention that there is not a “universal user” but a “universe of users” (Bowie, 2009, p. 135). Even within this group of people who were relatively similar in terms of factors such as race, class, gender, education level, geographical and cultural heritage, and disciplinary formation, perceptions of functionalities were extremely diverse.

As indicated in Figure 4, items are quite unevenly distributed across the categories. Some, including “Help” and “New User,” might conceivably be combined. The persistent toolbar did not survive as a category, perhaps as a result of lack of user familiarity with the concept. However, the idea of something like universally available functionalities circulated within groups in a range of ways, including in the invention by one group of the idea of a “megatool.” Search functionality seems very likely to establish itself as a persistent top-level feature, as perhaps too will Collaborate, by analogy with the “Share” button found in the menu bars of many socially-networked applications. They would thus arguably join the original rubric of Create, Read, Enhance, Analyze, and Manage. If we represent the twin Cs with a capital and the rest of the acronym with small caps, then (setting aside the anagram SCREAM as unhelpful) we arrive at CREAMS as a rubric under which to organize the next stage of sketching and prototype development.

Figure 4. Distribution of items by category

Provisional conclusions and future inquiry

The analysis presented here is far from exhaustive, and further analysis might well yield more insights. Work on primitives in the context of online collaboration, for instance, presents a framework within which it would be useful to situate these findings (Arazy, Stroulia, Ruecker, Arias, Fiorentino, Ganev, &Yau, 2010). Statistical evaluation of inter-rater agreement might also prove fruitful (Landis &Koch, 1977; LeBreton & Senter, 2008), leading to a clearer sense of how much divergence there is even within our self-selected user community.

What seems clear, however, even based on these preliminary results, is that neither CREAM nor CREAMS nor any other simple rubric will offer an easy solution to the challenge of designing an interface for a rich scholarly repository. The functionalities are very diverse, scholarly understanding of them is uneven and evolving, and it will doubtless take considerable further work, experimentation, and user feedback to figure out what makes most sense. CREAMS can be understood not as a prescriptive mnemonic, but as a provocation, a starting point for thinking about the design of scholarly research interfaces: this modified acronym can remind us simultaneously of the richness that needs somehow to be represented, the fluidity of its component categories, and the plurality of understandings that are involved.

This article began with the question of whether the concept of rich prospect applies to functionalities or just to collections of objects when considering interface design, and noting that this concept exists in tension with the principle of minimalism. Our results with prospective users of our complex interface suggest that the principles of organization of such interfaces are far from clear, and bears out the premise that the tension between richness and minimalism signifies a major challenge in the design of interfaces for a plethora of complex, inter-related functionalities that will be engaged differently by different people.

Furthermore, although this study was a preliminary one, that its relatively homogenous group of respondents produced such heterogeneous results is a clear indication that, although we have used here the dominant term “user,” the scholarly community that is partnering in the building of CWRC is composed not of users but, as Johanna Drucker has insisted, of “subjects,” with all that the term entails in the context of work in the humanities (p. 3). The onus is on us, as a community engaged in the production of interfaces for humanists, to draw on the tradition that has explored the complexities of the subject’s evolving and mutually constitutive relationships with environments. Such a perspective offers a corrective to “a mechanistic consumer model of the autonomous viewer,” (Drucker, 2011, p. 3) and pushes us to recognize that expectations of and responses to interfaces depend upon many factors including “embodied and situated knowledge, cultural conditions and training, the whole gamut of individually inflected and socially conditioned skills and attitudes” (Drucker, 2011, p. 6).

The tension between minimalism and richness, then, needs to be situated in relation to this larger frame. Yet even as our experimentation with this technique brings home the extent to which devising an affordance-rich scholarly interface will necessarily involve a multi-faceted and iterative process, it was a useful exercise in several ways. It brought home the immensity of the challenge we face. It offered some sense of where there seems to exist a community consensus on some affordances within the system, and where it seems to be lacking, so that we can focus on developing strategies that will address what may be conceptual, or at least terminological, indeterminacies that seem to surround categories such as enhancement. Last but not least, it involved the scholarly community that seeks to engage with each other and their work within the prospective CWRC environment in the challenge of the interface itself. This is important, because by and large scholars who research and teach through technologies are no more engaged critically with their political, ideological, and conceptual impacts than when Cynthia and Richard Selfe called for such analysis almost two decades ago, even though our work has become inextricably imbricated with them (Selfe & Selfe, 1994). Active involvement of the research community is fundamental to the design of the CWRC project, and this exercise proved an effective means of initiating the conversation about the interface as a whole, one that seemed more effective in evoking alternative ideas than soliciting responses to a sketch. In sum, it offered an illuminating starting point. For all the tension involved between minimalist design and a complex array of affordances, our community’s engagement with the challenges of interface conceptualization suggests strongly that approaching interface design for complex digital systems with a view to CREAM or, better, CREAMS — Create/Collaborate, Read, Enhance, Analyze, Manage, and Search — makes for a better start than CRUD. Without necessarily investing for the long term in any particular vocabulary or rubric, we can adopt CREAMS as a salutary reminder of the extent to which a multiple, fluid approach is demanded in approaching interface design for rich online environments from a humanistic perspective.

---

Website

Canadian Writing Research Collaboratory. [Website]. URL: http://www.cwrc.ca

References

Arazy, Ofer, Stroulia, Eleni, Ruecker, Stan, Arias, Cristina, Fiorentino, Carlos , Ganev, Veselin, & Yau, Terence (2010). Recognizing Contributions in Wikis: Authorship Categories, Algorithms, and Visualizations. Journal of the American Society for Information Science and Technology, 61(6), 1166–1179.

Bowie, Jennifer L. (2009). Beyond the Universal: The Universe of Users Approach to User-Centered Design. In S. Miller-Cochran & R. L. Rodrigo (Eds.), Rhetorically Rethinking Usability: Theories, Practices, and Methodologies (pp. 15–31). Cresskill, NJ: Hampton Press.

Brown, Susan, Menezes, Chris, Nonnecke, Blair, Ruecker, Stan, & Warwick, Claire (2009). Studying Orlando’s Interfaces. Paper presented at the Society for Digital Humanities/Societé pour l’etude des medias interactifs Conference, Humanities and Social Science Federation of Canada Congress. Ottawa, ON: Carleton University.

Brown, Susan, Clements, Patricia, Elio, Reneé, & Grundy, Isobel. (2006). Between Markup and Delivery; or Tomorrow’s Electronic Text Today. In Raymond Siemens & David Moorman (Eds.), Mind Technologies: Humanities Computing and the Canadian Academic Community, (pp. 15–31). Calgary, AB: University of Calgary.

Brown, Susan, Clements, Patricia, & Grundy, Isobel. (2006). Orlando: Women’s Writing in the British Isles from the Beginnings to the Present. Cambridge, UK: Cambridge University Press. URL: http://orlando.cambridge.org [October 31, 2011].

Carroll, John M. (1998). Minimalism beyond the Nurnberg Funnel. Cambridge, MA: Massachusetts Institute of Technology.

Chapman, Cameron. (2010). Principles of Minimalist Web Design, with Examples. Smashing Magazine. URL: http://www.smashingmagazine.com/2010/05/13/principles-of-minimalist-web-design-with-examples/ [October 31, 2011].

Drucker, Johanna. (2011). Humanities Approaches to Interface Theory. Culture Machine. URL: http://www.culturemachine.net/index.php/cm/article/view/434/462 [June 15, 2012].

Hackos, JoAnn T. (1999). An Application of the Principles of Minimalism to the Design of Human-Computer Interfaces. Common Ground, 9, 19–26. URL: http://www.comtech-serv.com/pdfs/Minimalism%20Human-Computer%20Interfaces.pdf [June 16, 2012].

Landis, J.R., & Koch, G.G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), 159–174. URL: www.jstor.org/stable/2529310 [June 15, 2012].

LeBreton, J., & Senter, J. (2008). Answers to 20 Questions About Interrater Reliability and Interrater Agreement. Organizational Research Methods, 11(4), 815–852. URL: orm.sagepub.com/content/11/4/815.full.pdf [June 26, 2012].

Nielsen, Jakob. (1994). Heuristic evaluation. In J. Nielsen, & R.L. Mack (Eds.), Usability Inspection Methods (pp. 25–61). New York, NY: John Wiley & Sons.

Nielsen, Jakob. (2005). Ten Usability Heuristics. URL: http://www.useit.com/papers/heuristic/heuristic_list.html [October 31, 2011].

Nowviskie, Bethany. (2007). A Scholar’s Guide to Research, Collaboration, and Publication in NINES. Romanticism and Victorianism on the Net, 47. URL: http://www.erudit.org/revue/ravon/2007/v/n47/016707ar.html [October 31, 2011].

Radzikowska, Milena, Ruecker, Stan, Brown, Susan, Organisciak, Peter, & the INKE Research Group. (2011, June 19–21). Structured Surfaces for JiTR. Digital Humanities 2011 Conference Abstracts. Stanford, CA. URL: http://dh2011abstracts.stanford.edu/xtf/view?docId=tei/ab-230.xml;query=;brand=default [October 31, 2011].

Rockwell, Geoffrey, Ruecker, Stan, & Ilovan, Mihaela. (2011). The Interface of the Collection. Digital Humanities 2011 Conference Abstracts. Stanford, CA. URL: http://dh2011abstracts.stanford.edu/xtf/view?docId=tei/ab-230.xml;query=;brand=default [October 31, 2011].

Ruecker, Stan. (2006). Experimental Interfaces Involving Visual Grouping During Browsing. Partnership: the Canadian Journal of Library and Information Practice and Research, 1(1), 1–14. URL: http://journal.lib.uoguelph.ca/index.php/perj/article/viewArticle/142/177 [October 31, 2011].

Ruecker, Stan, Rockwell, Geoffrey, Brown, Susan, Sinclair, Stéfan, & Organisciak, Peter. (2009, June 22-25) Mashing Texts: Supporting collections level text analysis. Digital Humanities 2009 Conference Abstracts. University of Maryland, USA. URL: http://www.mith2.umd.edu/dh09/wp-content/uploads/dh09_conferencepreceedings_final.pdf [October 31, 2011].

Ruecker, Stan, Radzikowska, Milena, & Sinclair, Stéfan. (2011). Visual Interface Design for Digital Cultural Heritage: A Guide to Rich-Prospect Browsing. Franham, Surrey: Ashgate.

Selfe, Cynthia L. & Selfe, Jr. Richard J. (1994). The Politics of the Interface: Power and Its Exercise in Electronic Contact Zones. College Composition and Communication 45(4), 480–504.

Spencer, Donna & Warfel, Todd. (2004). Card-Sorting: A Definitive Guide. Boxes and Arrows. URL: http://www.boxesandarrows.com/view/card_sorting_a_definitive_guide [September 29, 2011].

Unsworth, John. (2000). Scholarly Primitives: What Methods Do Humanities Researchers Have in Common, and How Might Our Tools Reflect This? Paper presented at Humanities Computing: Formal Methods, Experimental Practice. London, King’s College. URL: http://www3.isrl.illinois.edu/~unsworth/Kings.5-00/primitives.html [October 31, 2011].

Wheeles, Dana. (2010). Testing NINES. Literary and Linguistic Computing, 25(4), 393–403.

---

Appendix: Summary of card-sorting results

Analyze	Apply various analysis tools to articles in the collection
# items	3 (1 item cross-categorized)
ID	Item
VI	Analyze the words (frequency, collocation etc.) within a text or collection
XCIII	Visualize the social network embedded in a collection or set of search results
LXXVII	See a video file
Collaborate	Collaborating with other users of the collection
# items	10
ID	Item
XVIII	Communicate with another researcher in CWRC
XXXVIII	Find out who else is working in areas related to yours
LXXXI	Share material in progress with a colleague or group of colleagues
I	Accept/reject a correction by others of materials in your collection
XIX	Communicate with the CWRC team
XXXV	Establish a personal profile page outlining research interests
XLI	Get a report/visualization of how much a person has contributed to CWRC/a project
XLIII	Give another person access to private work in progress
LXI	Make others members of your project
XCII	Visualize the network of scholars working in CWRC
Create	Adding a new item to the collection, including a new article, defining a new user, or a new metadata tag
# items	15
ID	Item
II	Add a new item to ORCA
XXVII	Create a new document in CWRC
LI	Input a draft of a text
LXXXV	Start a project/create a project profile page
LXXXIX	Upload a scan of a primary text
XCIV	Write an essay, article, author entry, or topic entry
III	Add a person, organization, text name, or placename to the CWRC indices
V	Add bibliographical items to ORCA
XC	Upload materials from your computer
IV	Add a set of search results to an existing ORCA collection
XXII	Contribute a snippet of information about a writer who has no entry
XLIX	Indicate what collection you want your contributions to go into by default
LXIII	Publish an item in CWRC
LXXXIV	Start a new research collection
LXXXVIII	Turn a scanned image into text
Enhance	Make changes to existing items in the collection, e.g. add metadata to items in the collection or edit a text
# items	9 (1 item cross-categorized)
ID	Item
LIII	Label a name, place, organization, etc. in a text you are reading in ORCA
LXXXVI	Supply bibliographical source information
XVII	Comment on an image/audio/video file
XXIII	Contribute biographical information about a writer
XXIV	Contribute critical information about a writer
XXXIII	Edit a text
LIV	Label components of an image
LVIII	Link an assertion to a bibliographical item
LXXXVII	Supply provenance information about an image, audio, or video file
Help	Seek help/instructions on how to use the application
# items	3
ID	Item
XLII	Get help/instructions on how to use the CWRC site
LXVIII	Report a bug or problem with the CWRC site
LXIX	Report abuse of system
Manage	Administrative tasks, including DB administration
# items	10 (2 items cross-categorized)
ID	Item
XXVIII	Define a workflow (series of steps or actions) for a collection or project
XLVI	Indicate approval of annotation by another CWRC member/member of public
LXXXII	Sign off on a workflow stage
XI	Check amount of storage quota used/request more storage
XII	Check that a set of materials is ready for publication
XIII	Check the workflow status of a collection being readied for publication
LXXII	Roll back a change in a document
XXV	Correct a scanning error
LXII	Merge two collections
XXIV	Contribute critical information about a writer
New user	New user learning about the project or looking to join
# items	2
ID	Item
LII	Join CWRC
LV	t CWRC
Read	Reading contents pushed (or filtered) to users, including news, selected articles, and information about other users’ activities
# items	6 (4 items cross-categorized)
ID	Item
LXVI	Read CWRC news
XXXIX	Find out who the members of CWRC are and their research interests
XXXIV	Enter settings for a standing search (spider) for new developments to a topic on the web and within CWRC
XLV	Hear an audio file
XXXI	Do an advanced search on the ORCA repository
LXII	Merge two collections
Search	Searching and browsing items in the collection
# items	16 (4 items cross-categorized)
ID	Item
XXIX	Do a free-text search on ORCA repository materials
LXVII	Refine a search by type of material/collection/tags or metadata
LXXIII	Save a search
LXXXIII	Sort through the results of a search, manually deciding what to discard or keep
XXX	Do a search that includes materials beyond ORCA repository
XXXI	Do an advanced search on the ORCA repository
XL	Form a timeline from a set of search results
LXXVI	See a collection/set of search results laid out on a map
XXI	Consult the index of people, places, organizations or texts in CWRC
XXXIV	Enter settings for a standing search (spider) for new developments to a topic on the web and within CWRC
XLV	Hear an audio file
XLVIII	Indicate personal default search options such as historical period, collection
LXV	Read a text
LXXIV	Save search results for later use
LXXVII	See a video file
LXXVIII	See an image
Undetermined
# items	26
ID	Item
VII	Annotate an entire item with a note
VIII	Annotate part of an item with a note
IX	Announce an upcoming conference
X	Apply for membership/Recommend someone for membership
XV	Cite a CWRC bibliographical item
XVI	Cite a source outside of CWRC
XX	Consult a document’s version history
XXVI	Correct an error of fact
XXXII	Download/backup materials to your computer
XXXVI	Export a set of bibliographical items
XXXVII	Export a set of results to use with an external tool
XLIV	Have CWRC suggest markup/tags/labels for an uploaded or new document
L	Indicate whether you want to view public annotations by default
LXIV	Put an article for private use in your own research collection
LXX	Reset password
LXXI	Retrieve/Reset password
LXXV	Scrape the entire content of a website (take everything from all the pages below the home page) to form a collection
LXXIX	Send search results to a CWRC tool
LXXX	Set preferences for privacy re: your activity/contributions in CWRC
XCI	Use tools to visualize a text or the contents of a collection
XIV	Check where a document is in a workflow
XLVII	Indicate how your activities can be logged (do we let people be anonymous?)
LVI	Learn what’s hot in CWRC/what others have been doing
LVII	Link a person, place, organization, or title to the project’s collection of entities
LIX	Link to information about a person or other entity outside CWRC
LX	Login/Logout

 

---

CCSP Press
Scholarly and Research Communication
Volume 3, Issue 4, Article ID 040151, 17 pages
Journal URL: www.src-online.ca
Received March 22, 2012, Accepted March 22, 2012, Published May 27, 2013

Brown, Susan et. al. (2012). From CRUD to CREAM: Imagining a Rich Scholarly Repository Interface. Scholarly and Research Communication, 3(4): 040151, 17 pp.

© 2012 Susan Brown et. al. This Open Access article is distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc-nd/2.5/ca), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.