“Conversation at the Boundaries Between Communities”: An Examination of Tutor and Peer Review Effectiveness Based on Commenting Practices

Download PDF

Contents

Introduction

Contextualization

A Review of Existing Scholarship

Methodology

Results and Discussion

Conclusions

Acknowledgments

Works Cited

About the Author

Methodology

I evaluated the comments made by Writing Fellows and members of the Rose Writing Studio informed by Nancy Sommer’s notion that comments should prioritize the writing process, not its product (156). The stress on improving writing more generally, rather than the paper presented for discussion, prompts tutor-training manuals to emphasize the value of commenting on higher-order concerns such as ideas, organization, development, and clarity over lower-order concerns such as sentence structure, style, mechanics, and spelling. In “Providing Effective and Efficient Feedback,” Melzer and Bean develop a hierarchy of questions to be asked when examining a student’s draft. These questions begin with “Does the draft follow the assignment?” (303) and end with “Is the draft carefully edited?” (309). Hence, the topic of discussion in a comment must be considered.

Furthermore, a comment’s content and wording matter. As such, I take into account Daniel B. Willingham’s apt assertion that specificity is important: “The goal of feedback on papers is assumed to be the improvement of future drafts and the improvement of the writing ability of the student. The specificity of comments is a critical variable in providing effective feedback” (10). This is compounded by Sommers’s finding that students experience difficulty interpreting vague marginalia. Therefore, acknowledging that comments should be specific, I define a comment’s specificity as meaning that it is adequately particularized to the context of a student’s draft so as to guide them when editing their work.

In formulating my method of comment evaluation, I followed Melzer and Bean’s systematic structure and Willingham and Sommers’s emphasis on specificity to develop a Likert scale rating a comment’s effectiveness from 1 (very ineffective) to 5 (very effective). I then applied the scale to each of the comments made on a student’s paper. These papers include three drafts voluntarily provided by former members of the Rose Writing Studio. They contain comments made by both a peer reviewer, who was another member of the Rose Writing Studio, and an experienced Fellow, who functioned as a co-facilitator of the Studio. Each comment made by the respective groups, peers and tutors, was rated according to the scale, and the scores of each comment on the paper were added up and divided by the total number of comments. This produced an average numerical value of “comment effectiveness” for those made by peers and tutors and allowed the comments made by the two groups to be compared systematically.

I chose this method to analyze the comments in an unbiased manner. Because I was aware of whether the comments offered on each draft were written by a Writing Fellow or a Rose Member, adhering to the structured scale above encouraged a degree of impartiality as I judged the effectiveness of each comment. The scale also allowed me to analyze the data comparatively, examining the average scores for tutor and peer review on each draft to determine larger trends within the limited data.