Matthew Wizinsky
University of Illinois at Chicago
Matthew Wizinsky is Assistant Professor in Graphic Design at the University of Illinois at Chicago School of Art and Design, 929 West Harrison Street, 106 Jefferson Hall, Chicago, IL 60607. Email: mwizin2@uic.edu .
Like pretty much every other person living in the “developed” world of the twenty-first century, I spend an awful lot of time reading and writing tiny pixel-based letterforms on small, glowing screens. During the short amount of time we have been acclimating to this new mediation for the acts of reading and writing, it seems that almost all visible media forms have collapsed into a single experience: looking at and touching a little piece of illuminated glass.
What then becomes of the printed page, and how can it play an active role in our new digital modes of reading, writing, seeing, playing, working, and learning?
The Innovation Center at the University of Illinois at Chicago (UIC), in conjunction with the UIC School of Design, is currently conducting research into methods by which paper objects can interface with computers and microprocessors to control multimedia reading and viewing experiences within learning environments. More specifically, recent experimentation with the use of conductive inks for paper-based interfaces has shaped a focused new area of typographic exploration that seeks to blend the constraints of drawing functional circuits with the creation of legible letters. These explorations reveal potential for books and printed matter to be considered as touch surfaces to control operations on digital media – making the printed page simultaneously information and interface. As the reader touches individual letterforms or words, the finger closes a circuit that triggers different media events. Touch a letter to activate textual documents, images, videos, or audio. Trace the words on the page with your fingertip to expand a collection of digital media.
If this sounds fantastic, but also a bit vague, allow me to put it in more concrete terms. We are effectively creating paper keyboards by printing conductive ink onto paper. These keyboards take the form of legible texts such that each letter or word works like a button. When “plugged in” to the appropriate hardware and software, simply touching a letter or word on this piece of paper has the potential to do anything you can imagine doing with a keystroke on a computer. Dreaming beyond a single piece of paper, the implications of these prototypes suggest a method for making printed books the interface for interaction with any digital media.
This work blends research into materials and material applications, interactive hardware and software systems, and typographic design – all at the intersection of digital and physical media. This is not the work of scientists, engineers, or computer programmers, but rather designers seeking to expand our possibilities for merging digital and physical media within learning environments. As such, form and content are not seen as distinct or divisible entities but are fully intertwined.1 This particular effort is being undertaken in conjunction with the History Moves project – a multidisciplinary collaboration at UIC with the mission to create a mobile digital humanities lab, in this instance through the creation of a mobile history museum.
What does it mean to collapse printed and digital media into a single, unified learning experience? How does this kind of integrated media interaction merge known and new modes of reading, viewing, and learning by experience? How do legible circuits present new challenges and opportunities in the realm of typography? And how viable is this approach to merging printed and digital media on a large-scale, distributable model?
Interface design, and often design in general, is not always viewed as a significant research activity, nor are its findings seen as serious research contributions.2 However, the decisions required to translate nebulous concepts of new models for accessing textual or pictorial information are made manifest exactly by those tangible systems and artifacts that allow us to further analyze, question, and develop the theoretical conversations. These systems and artifacts are cognitive tools that expand or refine the discourse every bit as much as the textual documents (yes, even this one!) that hope to do the same. As a designer whose research is done through making, and the critical analysis of what is made, I proclaim this bias proudly. So, there is my stake in the ground: design as research has legitimacy – not just in service to academic research but as a parallel contributor.
In this article, I will converge three considerations at play in this design-as-research method: the issues this work speculates on regarding the stability and flexibility of traditional and new media forms; new areas of research this work reveals for book design and typography; and how this prototyping work can be duplicated. My goal is to demonstrate the theoretical and practical value of this research and to do so in an open source spirit: here are the steps, try it yourself and make it better!
Considering paper as an interface for digital multimedia environments begins with making some assumptions about the value of printed media. One of the hallmarks of the print medium – dating to the moment of the invention of movable type – is the stable repeatability in the production of texts.3 This stability, in combination with the capacity to reconfigure type into yet more stable and infinitely repeatable texts, brought about a sea change in the distribution of information and ideas. The index system of referencing any prior text in order to build upon, expand, counter, or refine a trajectory of thought, starts with the possibility of all previous texts remaining stable – shared among the world’s citizens as a consistent human record. On the other hand, as McLuhan ardently argues, the technology of typography has re-proportioned our senses and divides our social and perceptual capacities. He states that “this type of reduction or distortion of all experience to the scale of one sense only is in tendency the effect of typography on the arts and sciences as well as upon human sensibility” (McLuhan, p. 125). This reduction of sensibility produced by the “technology and social effects of typography incline us to abstain from noting interplay and, as it were, ‘formal’ causality, both in our inner and external lives. Print exists by virtue of the static separation of functions and fosters a mentality that gradually resists any but a separative and compartmentalizing or specialist outlook” (McLuhan, p. 126).
Digital media present a whole array of new flexibilities in the production and distribution of text, images, audio, and other stimulations, while also making it possible to blend sense perceptions. Expanding the potential for variability, we have to consider the multitude of screens, surfaces, and resolutions, plus the spatial and physical-sensory conditions within which we can access digital material. However great these new potentials are, that exactly repeatable visual-pictorial process does lose some of its exactitude. It is in these diverse processes of accessing digital media that the reading-viewing experience loses some of its stability. If in doubt, try to access a Web page that was built just ten years ago (still well within the 21st century!). The information – textual and pictorial – may well be intact, but I will be surprised if the viewing and reading experience is exactly as it had been originally intended. Browsers, hypertext protocol, hardware – all have changed rather significantly in a very short amount of time. All of this flexibility in reading-viewing opportunities offers a tremendous increase in the varieties and potentials of new visual experiences, but it also hinders the expectation of perfectly repeatable experiences. This is where the printed page and the book excel!
Utilizing print-based interfaces to operate on digital media brings the stability of the printed artifact and the flexibility of digital media into communion. Imagine a book printed with a variety of legible circuits across its pages, by which I mean words that also clearly create a button-like interface. Each touch of a word or letter can trigger any kind of media event as defined by the software and hardware to which it is engaged. This allows for a single book to operate an infinite array of media events, simply by adding to or reconfiguring the digital media or software used to access it. In this way, the printed page retains its stability and perfect repeatability (as it does so well), while the digital media performs its flexible fluidity based on simple rearrangements of command lines or databases (as it does so well!). Not to mention that the interface itself – a book – continues to be perfectly useful and valid as a media artifact, even outside of its use as interface.
In the example of a mobile history museum – for which this initial prototype is being developed – the model being built upon is that of the gallery guide. A single, folded piece of paper with general information on the project is also designed as a print-based interface with conductive inks. Within the galleries, a visitor can “plug in” this guide to any of the multimedia exhibits to scroll images, activate videos, listen to oral histories, or peruse various textual and archival documents. Insert pages printed with additional textual and pictorial information specific to the current exhibition, and possibly with additional print-based interfaces to activate specific exhibits, augment this gallery guide. Visitors are encouraged to keep the document as a self-contained artifact – a growing archive of their own making – and the key to unlocking new digital media in future visits. In these three functions of the reconsidered gallery guide, we see an ongoing interplay with the stability and flexibility of both the printed and digital media.
For decades now, we have seen computer vision at work every time we go to the grocery store. The clerk swipes our packaged goods emblazoned with bar codes over their scanning devices, which are specifically designed to read these codes. The nerdier among us may have spent some time trying to decode these lovely line patterns (guilty!). However, we’ve come to recognize these as tools for machine efficiency – to get us out of the grocery store quickly – not to tell us anything intelligible about the product.
More recently, we’ve seen Quick Response (QR) codes pop up on everything from advertisements to business cards to museum signage. Devices using approaches like augmented reality have demonstrated that printed surfaces can control on-screen activities. We’ve also seen the development and application of fiducial symbols – such as those used in the open source Reactivision4 project – as markers that can activate multimedia events via camera tracking and image processing. All of these existing and emerging methods of machines reading printed materials expand what is possible in physical and digital interactions.
However, all of these approaches rely on specially designed shapes, symbols, or markers for computer recognition that are effectively illegible to the human eye. We recognize a bar code or QR code as being something intended for machine consumption. We know what it is, but very few humans, if any, can translate the visual expression into the string of characters embedded in its logic. This is all great if you are okay designing your materials with something that is obviously for computer recognition only and not intended for human comprehension. However, the goal of printing legible circuits addresses this gap between human and computer vision by proposing a new strategy that allows the designer to make the interface also informational – form and content fully intertwined.
Throughout history, typography has been shaped by a blend of forces that are simultaneously historic, cultural, and technological. For even the non-typographically inclined, one of the most obvious distinctions to be made among typefaces is whether or not they have serifs. Thus, serif or sans serif typefaces. The serif itself is a design feature rooted in an early technology of letter making: chiseling letters into stone. In order to cleanly finish the stroke of a letter, the carver or engraver would turn the chisel on its narrow end and stamp clean, thin horizontal lines on the ends of the letter strokes. It took about two thousand years and plenty of shifts in typographic production methods for the serif to be reconsidered. In the late 19th and early 20th century, sans serif typefaces – made possible by the increasing precision of metal type punching and eventually photographic reproduction methods – became prevalent contenders but still have not (and likely never will) entirely usurp or replace serif letterforms. This highly visible example of technique defining the aesthetic qualities of letterforms seems to corroborate the famous observation by typographer Zuzana Licko: “You read best what you read most.”5 Repeated exposure makes for familiar patterns, but these patterns are culturally derived, not necessarily universal.
Regardless, the basic forms of the Latin alphabet have such durability that extremely far-flung variations of these characters are legible and recognizable as distinct letterforms even if they do not provide easy, continuous reading. As we have developed into a highly literate, textual, and visual culture, we have come to see letterforms even where they are not – in random patterns, happenstance configurations of natural or human-made elements, and so on. Through deep and relentless exposure, our familiarity with the general features of these forms has made them ingrained patterns on our minds – almost to the extent to which we recognize human faces in abstract patterns or images.
Creating legible letterforms that are also functional circuit switches – or letters that work as printed buttons – shapes a new area for typographic research by defining functional requirements specific to this purpose. Yet within these parameters, there are endless possibilities for developing appropriately meaningful and expressive designs. In this work, technical and aesthetic decisions are necessarily one and the same. For the typographic designer, this is an exciting, new, and wide-open field ripe for visual experimentation and play!
There are a few relatively simple steps to create a printed legible circuit interface. First, designing the legible forms. Second, printing these forms. Third, connecting the print-based circuit to hardware and software that will interpret a touch (closed circuit) as the event to activate digital media.
There are three primary technical requirements for creating circuit switches. Once these three requirements are met, it is up to the designer to determine how to make forms appropriate for their particular purpose.
For prototyping purposes, as well as for relatively large-scale production, screen printing seems to be the most appropriate option for producing and reproducing legible circuits on paper media. Screen printing has its own history in the realm of visual reproduction. It may not be as precise as offset printing, considering the act of applying the ink still involves some human touch. However, allowing for some inconsistencies and imperfections (humanity!), the repeatability makes relatively large-scale reproduction possible.
In initial testing, the conductive ink used was Electric Paint by Bare Paint.6 The consistency and ease of use of this product worked well with the screen-printing process. Even diluting the paint with small amounts of water to make for better printing consistencies did not seem to alter the conductivity or effectiveness in any remarkable way. The Bare Paint website is a great resource, providing many video demonstrations for setting up and applying their products. There are several similar products currently available; however, early success with this one meant no further experimentation was required.
There are many methods for connecting your printed legible circuits to hardware and software that will allow you to access and activate digital media. At the writing of this paper, the UIC Innovation Center continues to work on various prototypes involving simple clips, clamps, and other means for intuitively “plugging in” the page to the machine. In all of these efforts, attention is paid to maximizing the use of the printed medium as an artifact itself and the flexible installation or application of the connective hardware. In other words, you still want to hold a book in your hands, to be able to turn the pages and read the book. The iterative prototyping of this connective hardware seeks to merge intuitive book handling (reading) with physical spaces and objects that are display surfaces or spaces for experiencing the digital media – via projection of text, images, and video or activation of audio media.
In these prototypes, a very helpful piece of hardware for testing has been the Makey Makey.7 The Makey Makey is essentially a streamlined microprocessor using alligator clips to very quickly turn any object with some amount of conductivity into a key. This allows very rapid testing, in which the key-touch is the signal to trigger some media event. Makey Makey functions on the principle that the human body can close a circuit by connecting its positive and ground ends by simultaneously touching an object and the ground wire. Easily connectible to a computer via a USB cable that also powers the board, your tangible interface – in this case the legible circuit – is put into direct communication with your software.
What you do with digital media once a legible circuit is touched is up to you. Anything is possible, and the possibilities are endless. For prototyping purposes, the Processing8 programming language and development environment has been an invaluable resource. Once connected via Makey Makey, all you need to do is decide what happens when a particular key is touched: turn on a video, select an image to move or highlight, toggle text documents on and off, play an audio file, and so on. Supported by over one hundred user-generated, open source libraries and a vibrant online community, Processing makes for an excellent sketching and development tool for beginner and professional programmers alike. In conjunction with this project, a new open source Processing library called ExHib is currently in the works to offer a variety of digital media display interfaces that work with physical interfaces.9
This body of research into paper interfaces and legible circuits represents some crude first steps toward conceptualizing, prototyping, and testing the book as an active and integral interface for interactions with digital media. This paper documents the early phases of this development and shares the challenges and repeatable processes in the hope that additional projects be taken up to challenge, expand, and refine the ideas and potentials. Employing a design as research methodology, the formal qualities of the produced artifacts – including the design requirements of creating legible circuits such that form, content, and interface are unified – are on equal footing with the conceptual and technical inquiries.
Those little glowing screens we are so accustomed to reading and touching may go away. And, it may be sooner than we think. Digital media is already starting to become directly accessible through responsive environments, smart objects, and augmentation devices that put almost limitless data in direct contact with our bodies and senses.10 Exciting? Scary? Probably both. No doubt, these shifts present a whole new litany of dramatic changes in our social, cultural, and physical organizations. Regardless, there are many who believe the book will not disappear in this new physical-informational paradigm. I am one of them.
However, let us not relegate this medium to the sanctity of a rare, expensive peculiarity of ages past – the kind of object only wealthy or scholarly individuals might possess. Instead, let us keep the printed book alive. Let us consider all the ways this bastion of the human experiential record can continue to function as an active and everyday participant in our digital media landscape.
Aicher, Otl. (1994). Analogous and digital. Berlin: Ernst & Sohn.
Bringhurst, Robert. (1997). The elements of typographic style. 2nd Edition. Point Roberts, WA: Hartley & Marks.
Electronic Textual Cultures Lab, University of Victoria. (2007). HCI-Book: Interface. URL: http://www.hci-book.org/cluster/proto_interface.php [August 27, 2013].
Hansen, Mark B.N. (2004). New philosophy for new media. Cambridge, MA: MIT Press.
Kittler, Friedrich Adolf. (1999). On the implementation of knowledge. Readme! (nettime): ASCII culture and the revenge of knowledge. Automedia, 60–68.
Johnson, Mark. (2007). The meaning of the body: Aesthetics of human understanding. Chicago, IL: University of Chicago Press.
McLuhan, Marshall. (1962). The Gutenberg galaxy. Toronto, ON: University of Toronto Press.
VanderLans, Rudy. (1990). Interview with Zuzana Licko. Emigre 15. URL: http://www.emigre.com/Licko3.php [August 27, 2013].
CCSP Press
Scholarly and Research Communication
Volume 5, Issue 2, Article ID 0502155, 9 pages
Journal URL: www.src-online.ca
Received December 13, 2013, Accepted January 12, 2014, Published April 11, 2014
Wizinsky, Matthew. (2014). Touch Here to Begin: Paper Interfaces and Legible Circuits. Scholarly and Research Communication, 5(2): 0502155, 9 pp.
© 2014 Matthew Wizinsky. This Open Access article is distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc-nd/2.5/ca), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.