Paper Versus “On-Line” (Part 1. Analytical Summary)

This series of articles discusses a 1997 technical report on reading paper versus “on-line” documents. (For consistency with the report being evaluated, the authors’ hyphenated “on-line” is used throughout).

Despite continual advancements in display technology, paper persists as the preferred medium for reading and note taking.

Designers have long sought to isolate what exactly it is that makes paper superior to onscreen for the purposes of reading.

O’Hara and Sellen (1997) conducted a laboratory study to compare paper and on-line reading.

This paper discusses the experimenter’s principle claims, their methodology, and the conclusions they draw from their findings. It then presents the advantages and drawbacks of paper versus on-line usage within the scenario of the author’s workplace.  

Principle ideas/claims 

1. Paper is superior to on-line because it allows annotation while reading, navigation is simple, and readers can utilize spatial layout flexibly. These advantages afford readers deeper comprehension of text, permit perception of general document structure, facilitate plan summarization, and support ready cross-referencing to other documents.
2. Paper is the preferred medium for reading activity. Studies conducted in the 1980s concluded reading from screens was significantly slower than reading from paper. However, such findings offer sparse insight into why this is so:
   Is the issue one of resolution, a problem that will diminish in proportion to advances in monitor technology? Or, is the difference due more to fundamental aspects of the perceptual process that are yet to be pinpointed?
3. The study aimed to demonstrate how reading from paper compares to reading from screen (the authors refer to the screen as ‘on-line’).
4. Unlike earlier studies, this experiment emphasizes design results. The authors claim their study differed from typical laboratory comparisons in three ways:
5. It did not attempt to measure only one or two highly specific aspects of reading behaviour; its approach was circumscriptive and descriptive.
6. The subjects’ task was devised based on field studies of reading activity within organizations; hence, it is a realistic representation of real world reading practice.
7. Subjects viewed video recordings of their behaviour and gave comments; this expanded the experimenters’ understanding of the reading/annotation process from the subject’s viewpoint. 
8. The majority of preceding studies evaluating reading from paper versus onscreen concentrated on ‘outcome’ measures such as reading speed, proofreading accuracy, or comprehension. Few studies have been performed to elucidate ‘process’ differences, such as readers’ eye movement when viewing text, or how users manipulate and navigate on-line documents. Such ‘outcome’ measures yield little in terms of the key differences between paper and screen.

The experimental task had to be ecologically valid and reflective of the demands made on the presentation medium of the document. The authors maintain that since previous experiments did not employ such methods, their experiment would.

Methods

The reading process is determined by how and for what purpose a text is read.  The experimenters selected summary writing as the task, because summarization requires a relatively deep process: ascertaining general understanding from text – as opposed to the lesser engagement demanded by skimming or scanning. Winograd (1984) claims the strategies of summarization overlap many of those used in general comprehension, and thus offer a better test of the medium. Summarization necessarily engages reading with writing, and is therefore identifiable with reading tasks that the authors observed ‘knowledge workers’ performing on a routine basis. Testing based on this skill ought thereby to generate real world-applicable results. A key element of knowledge work is reading in conjunction with writing – summarization (in most cases, the two are simultaneous: annotation occurs during reading). Based on these reasons, for the experimenters’ purposes, summarization was the optimum test skill.

• Subjects were 10 laboratory staff members.
• Their task was to write a 200- to 300-word summary of a 4-page science article.
• Paper condition subjects read the article from paper and summarized it on paper.
• On-line condition subjects read it on-line and used on-line tools* to summarize it.
• Both sets of subjects could make as many notes as they wanted, mark the article however they pleased, and refer back to it as often as necessary.
• Post test, subjects were questioned about their performance. All sessions were recorded on video and subjects provided commentary on behaviour that was of interest to the experimenters.

* The On-line condition was deliberately handicapped. The experimenters chose not to use the most advanced display hardware and software available, but rather a more conventional system, i.e. an ordinary set up, a typical workstation that would be used to run a word processing application. Such employment of average display technology yielded insights into the benefits and drawbacks of a typical on-line reading scenario.

The Paper condition

Three documents in a pile – the article, a blank sheet for notes, and a blank sheet for the summary.

The On-line condition

Microsoft Word displaying three documents – the article, a blank document for notes, and a blank document for the summary. A colour screen displayed the documents in scrollable page layout mode so that they visually resembled sheets of paper.

Results

Both sets of subjects reported that note taking was important for increasing understanding of the text and planning the summary. Notes were a means of highlighting important points and making implicit the connections between them. Both these activities aided subjects in creating structure for the summary.

Paper condition subjects made notes while reading. Subjects marked the text and used these marks at later stages to refer back to important points. The marks facilitated rereading and emphasized key information (also found by Anderson and Armbruster, 1982). Subjects used their separate notes for subsequent reference also, but in a different manner to the annotated text: the separate notes helped subjects restructure and collate information. These notes developed into plans or outlines for the summary, which was enriched and modified iteratively by later annotation-guided rereading. Both forms of note taking were frequent and interleaved with reading, and both were performed without disrupting reading continuity.

In the On-line condition, only one subject attempted annotation of the document, and commented that marking the text as if it were paper belaboured the note taking process. Subjects wanted to keep their notes perceptually distinct from the original, so were reluctant to alter the source material. Such differentiation made for easier summarization, as subjects could refer to their own observations more readily.  Overall, On-line subjects did not attempt the annotation performed by the Paper condition subjects. None of the frequent back and forth referencing between notes and reading occurred as it did in the Paper condition.

Conclusions 1: Annotation

The ability to annotate while reading was critical to reinforcing understanding and planning the summary. In terms of annotation, three major differences were observed:

• Paper annotation was effortless and ad hoc, it integrated smoothly with reading, unlike the On-line condition, where annotation was a distraction from the reading.
• Subjects valued the annotation of the source document that was possible in the Paper condition. The On-line condition was comparatively inflexible, not supporting spontaneous marking and forcing undesirable tampering with the original.
• In both conditions, subjects made notes on separate documents: in the Paper condition, this was frequent and interleaved with reading; on-line, it was interspersed with long periods of editing, or notes were written after reading and with little referring back to the source document.

Conclusions 2: Navigation

Movement through the documents was critical to information organization, referencing, and checking understanding. In terms of document navigation, four essential differences were observed:

1. Paper navigation was quick, automatic, and interwoven with reading; on-line navigation interrupted reading, was slow and difficult.
2. Subjects could navigate two-handedly through paper and navigate while performing other activities; on-line navigation was one-handed, required effort, sometimes gave no feedback, and was spatially limited to certain onscreen areas only.
3. Paper provided tactile qualities that supported navigation, and users could estimate document length; on-line subjects failed to utilize explicit clues such as page length to estimate document length.
4. Information fixity in the Paper condition cued incidental memory for location of information, which helped subjects search and reread; on-line subjects struggled to view entire pages, but managed to use pictures as reference points. 

Conclusions 3: Spatial

The medium facilitated or hindered perception of document structure. Layout of pages in space was fundamental to subjects’ perception of overall document structure, both for referencing other documents and for integrating reading with writing. In terms of spatial manipulation, three essential differences were observed:

1. Paper: physically laying out paper allowed visualization of lots of information and creation of a ‘holding space’ for speedy reference to other documents; On-line: reducing documents resulted in loss of resolution and subjects were forced to use overlapping windows.
2. Paper: physical layout was flexible and dynamic, allowed easy cross-referencing and juxtaposition of reading and writing documents; On-line: subjects had to estimate window positions for future space requirements.
3. Paper: writing and reading spaces could be accessed concurrently and manipulated individually; On-line: only one window at a time could accept input, so reading and writing became serial activities, which subjects had difficulty integrating.

Discussion and Design Implications

This study shows that for the purposes of summary composition, paper is the superior medium.

Critical differences concern the major advantages that paper offers, namely supporting annotation during reading, easy navigation, and flexible spatial layout, all of which help the reader perceive document structure, plan summaries, cross reference other documents, and interleave note-taking with reading.