Kozma's study evaluates whether and how "computer based tools and embedded topical and rhetorical prompts," help college writers. He looked at tools that help the writer plan and outline compositions. His findings correlate interestingly with Oliver's examination of the effects of prompts. They both found an increase in the quality of writing containing rhetorical prompts, regardless of whether the writer was already skilled in their use. Kozma also found an inverse relationship between the quality of less skilled writers and the time they spent planning. More skilled writers increased their quality by spending more time in the planning phase. His study concludes with implications and suggestions for research, software design and composition pedagogy.
Robert Kozma's work concerning learning with media is widely cited in papers dealing with teaching, cognition and technology. In this 1991 study, he adds further evidence to studies by Butcher and Kintsch, and Oliver that rhetorical prompts play a large role in the quality of writing. Consideration of audience, topic, purpose and context play a role in the stages of the writing process including planning, drafting and revising by bringing the goals of writing continually into the process. These studies are interested in measuring and evaluating cognitive effects produced when rhetorical prompts are introduced. Strategy, structure and thoroughness introduced by prompts in the planning and drafting stages results in better writing.
One interesting observation is that these prompts produce better results with older and more experienced students while the use of them may confuse students who haven't been taught how to use them. For my project, this study raises questions about the skill level of the Christian composer asked to write their testimony. For example, how do they construct an audience and do they have the training necessary to change the message to adequately meet each audiences' expectations. What skills are necessary for a writer to convey very personal and difficult to express spiritual insights and experiences? Kozma shows increases in planning as an asset to the skilled writer, while a deficit to the unskilled. For my project, this study does not bring into focus the differences between writing and speech that are also a crucial consideration.
To study the effects of computer-based writing tools, Kozma designed a study using 41 students classified as either advanced or novice. Novice writers were "students enrolled in an introductory English composition course...," advanced writers "had taken at least two writing courses, one of which was in argumentative writing."(6) All students were trained in the use of MacWrite 4.5, a word processor. Some were also trained in an outline Acta, or an idea organizer Learning Tool.
Subjects were then given two hours to complete an argumentative writing task consisting of "... a baseline writing assignment, a practice assignment and a final assignment"(7). Subjects were individually measured for demographic differences and for "writing anxiety"(8). Finally, the subjects were evaluated on two kinds of "dependent" measures, "cognitive strategies..," and,"...the quality of the written product."(8) Cognitive strategies were measured by speaking aloud as they thought about constructing the writing. These were recorded, differentiated, and coded based on many evaluative points including writing, re-reading, use of media, planning, etc. The writing itself was measured by two "highly qualified and experienced raters,"(9) who used a rubric containing six points, three on quality and three on organization. These measures were all evaluated statistically, and not easily explained here.
I found the study well conceived and the measuring points and criteria very specific. The correlations were clear and yielded good information about how these software programs help students.
However, questions arose for me about the cognitive measurements using the "thinking aloud,"(8) method of recording cognitive processes. Kozma cited other examples for that method, but I wonder how well they reflect the actual writing process used by students. Awareness of having one's thoughts recorded has to alter the actual thinking, as does speaking. I think this could skew the results. This type of complex study is useful to understanding the interplay of mind, technology and writing, construction and application; however, this study requires interdisciplinary knowledge of writing, teaching and cognitive psychology.
