Contents Papers Visual design I Visual design II Evolving multimedia		Visual Design for the User Interface Part 2: Graphics In the Interface Patrick J. Lynch, MS Yale Center for Advanced Instructional Media Published 1994, Journal of Biocommunications, 21(2):6-15 Abstract Highly interactive multimedia electronic documents pose unique graphic information design problems. This paper is a discussion of some of the graphic design considerations that are unique to electronic documents. This article discusses the challenges of adapting existing graphic design skills to electronic documents that are displayed and read from computer screens. Sophisticated graphic information design skills are crucial in the creation of both computer software (tool-oriented software) and in the design of electronic documents (content-oriented software). Visual design skills affect virtually every aspect of software interface design, from the graphic details of the more generic interface objects like windows to the animated illustration of interactive three-dimensional structures within electronic documents (Marcus 1992; Schneiderman 1992; Tognazzini 1992; Tufte 1989). The following is a discussion of some of the factors that designers must consider when creating illustration and visual designs for graphic interfaces and multimedia electronic documents that will be used or read from computer screens. Characteristics of computer displays Graphics, text, and numbers printed on paper have been the dominant communications media in Western civilization for at least the last 500 years, and graphic design schemes for computer displays emulate many aspects of the style and organizational conventions of paper documents. However, computer displays in common use today share characteristics that make them very different from paper pages, and these differences pose new challenges to the illustrator and graphic designer. The first factors to consider are the shape and spatial resolution of computer screens. By convention most paper publications are oriented vertically, while most computer screens are horizontal rectangles. The small size and relatively low resolution of conventional office computer screens has lead most interface designers to avoid directly emulating vertical page layout grids used in paper documents. Most viewers expect to see the whole "page" on the screen at once. A vertical layout larger than the screen size requires the reader to constantly scroll around within a window without ever getting to see the whole "page." This is confusing, because it forces the reader to constantly remember objects that are not always visible on the screen, turning what should be an easy reading experience into a much more difficult task of remembering the nature and location of unseen information (Horton 1990; Norman 1993). The relatively low spatial resolution of computer screens and the small size of most computer displays combine to limit the complexity of screen graphics. Most current office display screens offer resolutions of 640 by 480 pixels (72 pixels per inch), or about 5,184 pixels per square inch of screen. Line artwork and typography in magazines printed on coated paper typically have resolutions of at least 1,440,000 dots per square inch (1200 dpi), or about 278 times the resolution of a typical computer screen. Using the technique of graphic antialiasing (see Figure 1) the apparent resolution of type and graphics may be increased on the computer screen. Reading tests have consistently shown that low-resolution text on the computer screen is less legible than high-resolution text on the printed page (Wright 1983). When reading antialiased text readers scores could be brought up to almost 98 percent of scores from paper documents (Brand 1987; Schmandt, 1987), but unfortunately system-level support for antialiased text is not routinely available yet in personal computers. However, designers should take very opportunity to render antialiased images as a means of increasing the legibility of the screen. For example, most complex illustrations, large type, and background graphics in multimedia programs are bitmap ("paint") graphics. Display type and illustrations rendered in PostScript illustrations programs like Adobe Illustrator (Adobe 1993d) or Aldus FreeHand (Aldus 1993) may be imported and converted to antialiased bitmap graphics with image edited programs such as Adobe Photoshop (Adobe 1993). This allows the same PostScript illustration to be used as both a high-resolution print graphic and an antialiased screen graphic for multimedia software. Although color or gray-scale diagnostic images, photographs, and illustrations rendered on computer displays also suffer in comparison to print, the differences in spatial resolution between screen and paper are less dramatic because offset lithography requires color or tonal images printed on paper to be rendered in halftone screens (typically 150-line) that are only a little more than four times the resolution of most computer screens. However, the lower spatial resolution of computer screens is partially compensated by the availability of hundreds or even millions of colors with which to render tonal images. Thus computer displays of gray-scale or color diagnostic images often compare quite favorably to halftone or four-color reproductions printed on paper. Figure 2 illustrates the how increasing the number of colors or shades of gray increases the density of information available from a given unit of display screen. Aside from purely aesthetic concerns, the primary advantage of a full-color display is the enormous increase in the depth and resolution of information displayed. In Figure 2 note how much high the spatial resolution of the gray-scale image is when compared to the black-and-white rendering. Although it is rarely mentioned in the literature on computer display screens, the transmitted light ("backlighted") rendition of images on the typical computer display screen is one of the major differences between reproductions of images on the computer screen versus the reflected light viewing of images on the printed page. The greater tonal range of transmitted light more faithfully renders radiographs, magnetic resonance images, and other diagnostic images that are typically read in transmitted light (from computer screen or from light boxes) in clinical setting. Designing for the Screen Information-oriented graphic design in user interfaces seeks a balance between the practical necessities of information management and the esthetics of presenting text and graphics to the reader. The graphic design of a program or multimedia document is built up through the systematic use of standardized interface elements such as symbols or icons, and interactive screen elements such as windows and buttons. These interface elements are organized over a design grid similar in structure and purpose to the design grids used to create paper documents. As in print, the primary goal of graphic design for the computer screen is to establish a consistent visual structure, in which the important information is immediately obvious, and where everything else is subordinate and undistracting. Due to the relatively low spatial resolution of display screens graphics must be robust and carefully crafted to match the screen's grid of pixels. Delicate graphics or typefaces that depend on fine details not readily visible on the computer screen should be avoided. In projects that incorporate large amounts of text it is especially important to establish a clear and consistent graphic layout for text blocks, as text is much more difficult than reading text from the printed page. Abrupt or arbitrary changes in the layout of interface screens will distract and disorient the reader. Users of multimedia computers documents don't just look at information, they interact with it in novel ways that have no precedents in paper document design. Excellence in interface design-designing how the user is able to access the information in your document-is crucial to the success of digital documents. The first design consideration in any multimedia project is to analyze the relationship between the visual and functional goals and requirements of the project and the standard interface guidelines of the operating system (Macintosh or Windows, for instance) the project will run under. Unfortunately, inexperienced multimedia designers often ignore or are simply not aware of the existing interface design guidelines for the major graphic interfaces in use today, and thus impose a large burden on themselves and their users by attempting to re-invent graphic interface design with every new multimedia project. If you follow standard interface guidelines the vast majority of you users will easily "look through" your interface and concentrate on what's important-the information your document presents. Many first-time designers come to process of graphic interface design committed to the goal of producing "friendly" easy-to-use software screens without balancing this against the more important goal of enticing the user with sufficient depth of content. Good interface design does not require barren screens that hesitate to provide the user with more than one or two choices at a time. While the needs of naive users are always a concern, truly naive users are increasingly rare in personal computing. Most undergraduates, nursing students, medical students, and physicians now have some experience with graphic computer interfaces and are typically more concerned with depth of content. These routine users of personal computers are likely to be annoyed by a patronizing level of "user-friendliness." By working within standard interface guidelines very complex interfaces may be built without requiring experienced users to re-learn a whole new approach to using their computers. This aspect of the visual interface design may have a crucial impact on the financial and critical success of a publication. Following interface guidelines closely should result in many fewer user support problems once the software is published. The dominant graphic user interfaces in use today (Macintosh Operating System and Microsoft Windows) have well-documented user interface standards that should be carefully considered during the initial design phase of any project for the computer screen (Apple Computer 1992; Microsoft 1992). Although all the major graphic interfaces incorporate basic elements like windows, menus, dialog boxes, or screen buttons, each of the interfaces have particular visual design conventions that should be carefully considered and should form the foundation for the project's design grid. As in print documents, electronic documents are structured over a design grid. This "backbone" establishes where the major blocks of text and illustrations will regularly occur on the screen, and where the major screen titles, navigation buttons, and other essential elements of the screen design will sit on each of your screens. Consistency in the screen design grid is especially important because of the interactive nature of the computer interface, where visual design elements appear and disappear in response to the user's actions. The screen grid organizes this shifting temporal quality of the screen, so that objects like windows, buttons, graphics, or text fields appear in stable, intuitive, and predictable relationships one another. The layout grid must also accommodate and standardize the locations and visual design of status indictors like special symbols or warning signs, and should define standard methods or screen locations for providing text messages and other feedback to the user. The appearance and disappearance of graphic interface elements may be thought of as a constantly shifting interactive animation. As with any animation the accurate placement of screen elements is crucial. Any slight mismatch in the location or size of interface elements as they appear and disappear will cause annoying inadvertent "animations" as the graphics shift, flicker, or wiggle in and out of registration. The design grid must accommodate both the most and least complex layouts within the program, and must be flexible enough to allow the inevitable modifications and evolution that occurs during the construction of large projects. In large multimedia projects it isn't possible to predict in advance how every particular combination of text , graphics, audiovisual material, and interface elements will interact on the screen. The goal is to establish a consistent substructure, one that allows you to easily "plug in" text and graphics for each new screen without having to stop and re-invent your layout approach with each new design problem. Without a firm underlying design grid the screen layout will be driven by the difficulties of the moment, and the overall design may look patchy and confusing to the user. Typography for the computer screen Typographic design for the computer screen is difficult because of the relatively low resolution of personal computer displays. The low-contrast of reflected light LCD screens now used on many types of portable computers also severely limit type legibility in all but the best lighting circumstances. These compromises in the resolution and visual contrast of screen typography result in reduced reading speed and comprehension (Gould 1986), but proper typographic design can do a great deal relieve the difficulties of text in computer documents. Although most standard type faces have been adapted for use in computer desktop publishing systems these faces are still primarily designed for relatively high resolution (300 dots per inch or more) use on paper. The screen fonts that accompany most PostScript typefaces often provide little more than rough previews of what the printed text will look like, and are often almost illegible on the computer screen at small point sizes. Scaleable typographic display software such as Adobe Type Manager (ATM, Adobe 1993a) or TrueType (Apple 1993; Microsoft 1993) greatly improves the rendering of larger type sizes on computer screens, but neither ATM nor TrueType software improves legibility at standard text sizes (9-12 points), and may actually decrease the legibility of some decorative or complex typefaces rendered in small point sizes on screen. Type faces used in electronic documents should be always be judged solely by their appearance on the computer screen, and not by the esthetics of a particular font as printed on paper. The low resolution of screen displays often severely compromises the legibility of condensed, light, extended, or decorative typefaces. In general typefaces designed for maximum legibility in low resolution displays have relatively large x-heights, simple character shapes, and each of the smaller font sizes (9, 10, 12 points) is specially optimized for legibility on the screen. Proportionately spaced typefaces such as the standard Macintosh screen face Geneva or New York (Apple computer 1993), MS Sans Serif or MS Serif (Microsoft 1993), or Adobe's Stone and Lucida PostScript typeface families (Adobe 1993b) are were specifically designed the be legible in a wide variety of low-resolution media. Because the typography within an electronic document currently depends on the typefaces present in the operating system of the computer the designers of electronic documents face problems unique to the computer screen (Brown 1991). Typographic layouts on the computer screen are created anew by the computer's operating system each time a document is displayed on the screen, using font resources from the operating system and the text fields built into your electronic document. Because the typography is dependent on variable factors like the presence or absence of specific fonts within the system, or the presence of font-scaling software like ATM or TrueType, the precise layout of your text blocks, titles, and other screen type may vary. Usually text in long text blocks will not shift too radically in position or legibility, but text in small labels or tightly-cropped titles may be drastically affected by even small changes in font geometry is you do not leave enough space within the text field for the text to expand slightly. If tightly-cropped text is displayed in slightly expanded form the text may not fit into the field, and either drop down to the next line, or simply disappear because it no long fits in the text field. Although almost every popular typeface in current graphic design is now available in PostScript or TrueType form for the computer screen the choices available to electronic publishers are often limited by practical concerns to just those type faces supplied by the operating system manufacturers with the system software. If you chose a typeface not present in the standard set supplied with the operating system the typeface must be supplied with the electronic document, and this may require the user to take extra steps (which they may or may not bother with) to install the font into the operating system. Prior to Macintosh System 7.1 it was possible to install Macintosh screen fonts into the resource fork of HyperCard or SuperCard documents, but recent changes in the operating system's handling of font resources has removed this option. If the font you have chosen for the text in your electronic publication is not present in the operating system another font will be substituted, often with disastrous effects on precise typographic layouts. However, the recent advent of highly scaleable multiple-master font technology will make it possible to a standard set of built-in fonts to (approximately) emulate the look and geometry of all but the most complex or decorative typefaces (Fenton 1992). Currently the growth of electronic publishing is crippled by the lack of wide accepted file formats that can be used on many different types of computers or digital audiovisual display hardware. New cross-platform software technologies are now becoming available to standardize the layout and typography of screen documents, regardless of the computer or operating system in use. Adobe's Acrobat software uses the PostScript page description language and a new file format (Portable Document Format, or PDF) to allow documents with sophisticated layout and bit-mapped or object-oriented graphics to display properly on many different computer systems, even if the systems lack the proper fonts to display type (Adobe 1993c). Future version of Acrobat will allow complex text search and retrieval, and incorporate digital sound and video information. While this will solve some of the practical problems of electronic publications, designers who require specific typefaces within complex typographic designs will continue to face difficulties until the designers of computer operating systems address this issue. When in doubt, restrict your typefaces to those built into the operating system of the computer. At standard screen reading distances text blocks should be set at a minimum of 9 points, with relatively generous leading (9/13, or 9/14) to improve the legibility of small print. Increased line spacing, especially when used in 10 or 12 point sizes can greatly improve reader speed and comprehension (Human Factors Society 1988). HyperCard (Apple Computer 1991), SuperCard (Aldus 1991), and most other multimedia authoring tools allow several tricks that can increase the legibility of long blocks of text on the computer screen. Leading (or "line height" as it is called in HyperCard) is a traditional typographic term referring to the amount of space between each line of type. HyperCard allows you to easily control the amount of leading in text fields. A good rule of thumb in smaller text sizes is to use a "line height" or leading 3 to 4 points greater than the size of the type. For example, for 12 point type choose 15 or 16 points of leading. The extra line spacing makes scanning across the lines of type easier on the eye, and can actually make the type appear larger than it really is. The standard type controls in multimedia authoring programs also allow you to slightly expand the horizontal dimension of a typeface by selecting the "Expand" box in the list of type style options. Slightly expanding the horizontal dimensions of a suitable screen type face is a good trick for making thicker or boldface fonts more legible, and can sometimes make very small type sizes a little easier to read. Many of the typographic rules and standards used conventional print media also apply to text layouts for computer screens. The optimum line length for a text block should always be proportional to the size of the type font in use. Column widths of 40 to 60 characters provide optimal line lengths for legibility (Horton 1991). If the column length is too short relative to the size of the text there will be inadequate room for proper word spacing. Many multimedia authoring systems provide a "wide margins" option as a standard visual option when formatting text fields. Because text in computer systems is often contained within windows that may be dynamically resized and moved about the screen, the margins around a text block should be generous enough to provide adequate visual separation between the text field and other background objects. Spatial relationships in the interface Our day-to-day visual experience conditions us to believe that dynamic, interactive objects naturally exist three-dimensional space, and have logical visual and spatial relationships to one another. In graphic user interfaces screen objects show their spatial relationships by overlapping one another, scrolling off the screen "somewhere else," cast drop shadows, change in size, zoom in magnification, and objects sometimes appear to distance themselves from the viewer by "graying out" when inactive. We have no real-world experience with such highly dynamic objects that exist only in the two-dimensional flatland of the computer screen, and so interactions with interface objects seem to occur within a special spatial framework unique to graphic interfaces. From the visual designer's point of view the computer screen may be best though of as a representation of a shallow space. In this form of graphic representation objects have a definite spatial relationship to one another, and their graphic representations should consistently reflect the spatial relationships implied by the interaction of "near" and "far" objects. Most multimedia documents also "layer" information related to the current topic by hiding windows or other objects until the user requests that specific information (see Figure 3). In current graphic user interfaces the visual layering techniques are limited and relatively crude; superimposing objects is still the primary means of visual layering in graphic interfaces The illusion of three-dimensional space is limited, and results in complex, confusing screen displays when many windows are layered in multitasking environments like the Macintosh (Fig. 3, right side). Depth cueing by varying the color saturation of objects has been used for centuries by landscape artists to suggest great expanses of space within painting. The painters noticed that the atmosphere causes faraway objects to lose color saturation and visual detail, and that mimicking these natural visual effects could produce paintings with the illusion of great distance. This same effect is useful for visual layering in the user interface, where important "foreground" objects should be rendered in strongly contrasting hues. Background or unimportant elements may be visually shifted back away from the user by rendering them in desaturated colors of shades of gray. Gray backgrounds are often gentler on he eye than the bright "paper white" backgrounds now seen in both the Macintosh and Windows graphic interfaces. Three-dimensional spatial representations also complement and reinforce interface metaphors. Users assume that interface elements rendered as three-dimensional objects will function much like their real-world counterparts. Thus control panels, buttons, and other interface objects meant to be "manipulated" or clicked on are often rendered as if they existed as real three-dimensional objects (see Figure 4). A simple line rectangle on the computer screen is spatially and functionally ambiguous; if the same rectangle is given a beveled edges and cast shadows like an physical button the user is much more likely to treat the "virtual button" as a dynamic, interactive part of the graphic representation on the computer screen. Symbols and semiotics in the interface Graphic symbols have long been used by industrial designers to suggest the function controls and buttons in a compact, nonverbal manner (Meggs 1989). User interfaces are dominated by nonverbal symbols, icons, pictograms, and other graphic metaphors, and the design of visual interfaces usually involves creating new symbols or adapting existing ones to represent standard actions and elements within the program. Figure 5 illustrates a typical screen from a multimedia program designed as a teaching resource for cardiothoracic imaging. The row of three button icons at the lower right represent standard features of the interface such as opening a magnified "zoom" window of the radiograph, showing labels on the radiograph, and opening a window with case notes. Semiotics, the study of signs, symbols, and their use and meaning (Eco 1976) has had a powerful impact on the designers of graphic symbols and typography. Semiotics has been advocated as a analytical tool and guiding principle in the visual design professions (Ashwin 1989) and particularly in the design of graphic user interfaces (Marcus 1992). The careful, systematic application of graphic symbols and icons is crucial to the success of interface design (Schneiderman 1992). Graphic interfaces and other symbolic design systems are not inherently "user-friendly," and a poorly designed interface can rapidly deteriorate into an elaborate digital rebus of cryptic symbols in ambiguous relationships (Lupton 1989; Nelson 1990). Summary Digital media are not just audiovisuals played on computer screens. Complex electronic documents may be a mixture of all forms of communications media, delivered in a highly dynamic, interactive form that challenges many of the basic tenets of audiovisual and graphic design. The most powerful forms of electronic documents are those that incorporate "live" data links from high-bandwidth optical fiber computer networks such as those now being assembled in the High-Performance Computing Initiative. These multimedia books and magazines will challenge the very definition of "publication" because they will not contain static content nor will they exist in multiple copies. There may be only one copy of most "publications" in existence, and all access to these on-line publications will flow from source to user through digital networks. Documents that incorporate live network links will dynamically change their content over time, in response to built-in instructions to seek out specific information from network sources at frequent intervals. Documents that attempt to assemble multiple forms of information that is in continual flux will be the ultimate challenge for graphic designers and audiovisual professionals. At the level of individual hospitals most medical records and patient information will eventually exist as live documents that continually search the both the local hospital information systems and national data sources such as Medline for information relevant to the care and treatment of that particular patient. As digital multimedia documents and publications become the dominant communications medium in the biomedical sciences, the design and production of and electronic documents will soon dominate and perhaps subsume many of the audiovisual professions within biomedical communications. This does not mean that the core skill disciplines of biocommunications will disappear-but the advent of such a powerful new communications medium will have profound effects on both the way we practice our individual disciplines, and create the need for new biocommunications professionals whose primary function is to design and produce digital multimedia documents and publications. Although most of the fundamental audiovisual design and production skills are equally applicable to these new media, the highly interactive nature of electronic documents will make human interface design an essential skill in biocommunications. Literature Cited Adobe Systems, Inc. 1993a. Adobe type manager 3.6. Mountain View, CA: Adobe Systems, Inc. Adobe Systems, Inc. 1993b. Adobe Type Library. Mountain View, CA: Adobe Systems, Inc. Adobe Systems, Inc. 1993c. Adobe Acrobat 1.0. Mountain View, CA: Adobe Systems, Inc. Adobe Systems, Inc. 1993d. Adobe Illustrator 3.2. Mountain View, CA: Adobe Systems, Inc. Aldus Corp. 1991. Aldus SuperCard 1.6. Seattle, WA: Aldus Corp. Aldus Corp. 1993. Aldus FreeHand 3.11. Seattle, WA: Aldus Corp. Ames, P. 1993. Beyond paper: The official guide to Adobe Acrobat. Mountain View, CA: Adobe Press/Hayden Books. Apple Computer, Inc. 1991. HyperCard 2.1. Cupertino, CA: Apple Computer, Inc. Apple Computer, Inc. 1992. Macintosh human interface guidelines. Reading, MA: Addison-Wesley. Apple Computer, Inc. 1993. System 7.1, Macintosh operating system. Cupertino, CA: Apple Computer, Inc. Ashwin, C. 1989. Drawing, design and semiotics. In Design discourse: History, theory, criticism., ed. V Margolin. Chicago: University of Chicago Press. Brown, A. 1991. Type renaissance: A primer on digital type. Macworld 8(7):203-209. Eco, U. 1976. A theory of semiotics. Bloomington, IN: Indiana University Press. Fenton, E. 1992. Understanding multiple masters. Macworld 9(9):264-268. Gould, J. 1986. Why is reading slower from CRT displays than from paper? In 30th Annual Meeting of the Human Factors Society in Santa Monica, CA, Human Factors Society, 834-835. Horton, W. K. 1990. Designing and writing online documentation. New York: Wiley. Human Factors Society. 1988. American national standard for human factors engineering of visual display terminal workstations. (ANSI/HFS 100-1988), Washington, DC: Human Factors Society. Lupton, E. 1989. Reading isotype. In Design discourse: history, theory, criticism., ed. V. Margolin. Chicago: University of Chicago. Marcus, A. 1992. Graphic design for electronic documents and user interfaces. New York: ACM Press, Addison-Wesley. Microsoft Corp. 1993. Microsoft Windows 3.1. Redmond, WA: Microsoft Corporation. Meggs, P. B. 1989. Type and image: The language of graphic design. New York: Van Nostrand Reinhold. Norman, D. A. 1988. The psychology of everyday things. New York: Basic Books. Norman, D. A. 1993. Things that make us smart: Defending human attributes in the age of the machine. Reading, MA: Addison-Wesley. Schneiderman, B. 1992. Designing the user interface: Effective strategies for effective human-computer interaction. 2nd ed., Reading, Mass.: Addison-Wesley. Tognazzini, B. 1992a. Tog on interface. Reading, MA: Addison-Wesley.