First, let me apologize to my fellow bloggers at PEA Soup for taking so long to break my silence. I agreed to join a couple of days before leaving for an extended vacation on the East Coast, and upon my return, the great weather we have had here made several home improvement projects shoot to the top of my priority list. But now that fall quarter has begun, my mind is back to philosophy and all that it has to offer.
Specifically, I have been thinking about my courses and the assignments I give to my students. I believe that a philosophy course must have an argumentative paper at its core, but I have been frustrated in the past with the papers I receive from students. So last winter, in an effort to improve student writing, I assigned a peer editing project to my students. Students were required to write a rough draft of their paper, and exchange these drafts on a specific date with other students in the class. They were then required to edit the papers they received from their classmates, return them to the authors, and make changes to their papers based on these comments.
Some students found it be an extremely challenging and rewarding assignment. One student claimed that it was the hardest assignment of her college career; another student claimed that she finally understood and could appreciate just how difficult our jobs can be; almost all students could not believe how poorly the others students wrote. Having them come to that realization was, of course, one of my main motivations in the assignment. Most students cannot identify the problems in their own writing; hopefully, by seeing how other students write, they can begin to identify the mistakes in their papers.
Overall, however, the results were mixed. For the students who put a lot of time and effort into the assignment, the assignment helped them to improve their writing. Others, however, did not put much effort into the assignment (e.g. one student claimed that every paper he edited was “Awesome” and the arguments “Rocked”. That was the full extent of his comments.)
So I have two questions for you: (1) Has anyone else assigned a peer editing project? If so, how were your results? I worry about repeating the assignment for this reason: if the people who edit your paper do not take the assignment seriously, then you will not recieve good feedback on your paper, and will not have the opportunity to improve the paper. This seems to punish you for the mistakes of another. (2) Do you have any suggestions for the logistics of grading the editing project? For example, how much weight should the editing project have relative to the paper itself? I find myself in a bit of a dilemma on this one. If I give the projects little weight, then few students put in enough effort. But I have a hard time giving the assignment a lot of weight because I wanted to grade it based largely on effort. What I found is that it can be remarkeably hard to judge the amount of effort expended, so to give the assignment a lot of weight was morally dubious.
Any suggestions would be welcome, for I am always looking for ways to help my students improve their writing and enjoy my classes.
Scott,
I’m going to be experimenting with just this sort of assignment this Fall. Can I ask whether you used a peer-review form where students are required to answer specific questions about the paper. I’ve decided to adopt one that asks the following sorts of questions:
“For each of the drafts assigned to you, provide hand-written comments on the drafts as well as typed responses to the following questions.
1. Is the philosophical issue well defined in the paper’s introduction? What is the issue? Is it clear why this issue is important? Explain why the author thinks that it’s important?
2. Can you identify a thesis statement? If so, state it. Does it appear in the introduction? If not, where? Is it clearly stated? If not, give examples of what you think the thesis might mean.
3. Does the introduction provide a preview of what’s to come? Explain.
4. …”
I think that there are a number of advantages to such a form. First, students can’t just say the paper is awesome and do no work; note that these questions force the peer-reviewer to do some work even if the paper is awesome. For instance, they have to state what the issue is and state what the thesis is. Second, these questions really focus the peer-reviewer’s attention on the sorts of things that I’m looking for in a paper. So, hopefully, when it comes time for them to revise their own papers, they’ll know exactly what I want from them. Of course, I plan on encouraging them to use these questions to assess their own papers before turning it over to their peer-reviewers.
Also, I tell my students not to make corrections. First of all, many of them aren’t good enough writers to make accurate corrections. Second, I don’t think that the peer-reviewers should be doing the work for the authors.
Here’s what I tell them in the directions for the peer-review assignment:
“Don’t try to rewrite anything; your job is to not to help the author rewrite the paper but to point out what the writer needs to revise.”
“Put an asterisk (*) next to each line where you think there is a grammatical mistake, spelling error, or awkward phrasing. If you are not sure, put a question mark after the asterisk (e.g., *?). Do not identify these errors or correct them. That is part of the author’s revision process.”
I’m very interested it what others are doing. I hope that we get a lot comments on this, so that I can avoid some pitfalls before implementing this assignment a month from now.
Doug,
Thanks for the suggestion to use the questions. Although I did use a list if questions, I think I had not planned well enough to avoid the one word answers. The examples you give (especially about identifying the thesis and the reasons supporting the thesis) are really good.
What I found was that some students had really long answers to the questions, but the answers were almost all fluff. I had a hard time determining if the student did not put any work into it, and so just wrote a lot of fluff, of if the student tried really hard, but just isn’t good at assessing arguments.
Hello Scott. Yes, I’ve used what what you’ve called peer-editing, with somewhat mixed results. My goals were similar to yours, in that I think the hardest thing for students to master about writing essays (philosophical ones in particular) is mastering writing for an audience. Many students, I think, operate on the idea that they are writing for the instructor or for themselves, rather than attempting to craft essays that meet objective disciplinary standards. So I’ve used these assigments in the hope that they will help students to develop a sense of these standards by considering their writing from a perspective besides their own.
Some particular concerns:
1. I’ve only used this technique in upper-division courses whose enrollment is mainly philosophy majors. I’m relucant to try it in intro or a similar course, because I worry that less experienced students just getting the hang of college writing may produce work that too’s rough even to be subjected to meaningful peer review or may not get much from such a high-level exercise. And to be honest, since many of my lower-division students just want to pass, they’re not likely to give much effort to something so intensive.
2. Like Doug, I think it’s important to give a good deal of guidance to the students, with specific objectives that the feedback has to meet. I don’t recall exactly what I’ve required from those giving feedback, but I know I asked them at least to describe the paper’s thesis, identify the paper’s strengths and weaknesses, and suggest directions for improvement.
3. I grouped my students in groups of about 4, and since in this case, they could choose from about five term paper topics, I grouped them according to topic, figuring that would improve the quality of the suggested edits and revisions. Still, differences in ability, effort, and interest led to widely varying levels of quality in the feedback.
4. Grading is a tough issue. Frankly, some of my students just flaked, which produces a lot of problems. I required that all final term papers had to include their peer reviews, along with a brief statement of how the author intended to revise the paper in light of the feedback. I didn’t grade at all based on the quality of the feedback itself, which minimized somewhat the differences in what sort of feedback students got.
5. You have to plan out your calendar very well, indicating the dates when drafts are due to the peer editors, when the drafts are due back to their authors, when the final drafts are due, etc.
On the whole, my own experiences were OK. Some conscientious students felt they gave more effort than the less conscientious, and some got very little useful feedback. I wonder if the best environment for this might not be in courses (with mostly seniors or majors, etc.) where you can expect a high level of commitment. I think it succeeded at least for some students in helping them acquire some intellectual distance from their own work. I’m extremely commited to that goal, and I think that assignments like this can really help. I’m still working on the details, so perhaps we can all try this and see what we learn.
Here’s a suggestion about the problem Scott describes in this paragraph:
“If I give the projects little weight, then few students put in enough effort. But I have a hard time giving the assignment a lot of weight because I wanted to grade it based largely on effort. What I found is that it can be remarkeably hard to judge the amount of effort expended, so to give the assignment a lot of weight was morally dubious.”
One thing you could do is hide the amount of weight from the students. For instance, you could say that 50% of their grade for the course comes from the whole project, and say the “project” is the peer editing they do plus the final draft of their own paper. In that case they might operate on the assumption that the peer editing part of the “project” is a big part of the grade, so they would expend a lot of effort on it. But when it actually comes time to assign a grade for the entire project you could make the peer editing a small part of it, so you wouldn’t be caught in the “morally dubious” position of assigning a lot of weight to the hard-to-judge quantity of effort.
David,
Inevitably, someone will ask how much each part of the project is worth, which will put the professor in the uncomfortable position of having to refuse to answer. I don’t believe it’s right to hide that information from students, or worse, to mislead them about their grades. The end does not justify the means here.
Scott,
I have to say, if it’s remarkably hard to judge the amount of effort expended, then you probably shouldn’t grade based on effort.
Ok. Enough moralizing. Here’s a suggestion: if the paper is a major part of the course, then perhaps you could supplement the peer editing with your own editing. That way, you can ensure that everyone receives good feedback.
It seems to me that many of us are unduly worried about the possibility that some students will not get good feedback on their papers because their peer-reviewers either lack the requisite skills or will fail to put in the requisite effort. I don’t think this should be a major concern, for the main point of the assignment is not for students to learn from the comments they get from their peer-reviewers, but for students to learn from the experience of reviewing the work of their peers. It is in the process of reviewing the work of their peers that students come to better understand (1) what the standards of a good philosophical writing are, (2) what the major potential pitfalls are, and (3) how to better identify faults in their own writing. If I’m right about this, then those who put the most effort into the peer-review assignment will get the most out of the assignment even if the quality of the comments they receive from their peer-reviewers is quite poor. In fact, as any professor who has commented on their students’ rough drafts will undoubtedly know, it is shocking how ineffective such comments (even if they’re from the professor) are in getting students to improve their writing. This is why I have decided not to waste my time commenting on students’ rough drafts anymore and am hoping that such peer-review assignments will prove much more effective in getting students to improve their writing skills. I have already had some success with some less ambitious peer-review assignments and am optimistic that the sort of ambitious peer-review assignment that Scott’s been using can work.
I agree with Michael that this kind of assignment is best for upper division students. Even some of these students lack the skills and motivation to do well on it, and I suppose that these are the students who are causing problems for me. I also agree with Jeff that not telling students how much each part of every assignment is worth is not a good idea (it is, in fact, the surest way to get students to hate you). Making my expectations and criteria clear is a necessary first step to doing a good job.
Doug–I am not sure I can refrain from worrying about the quality of the feedback a student gets. I agree that many students do not improve their writing after reading my comments, but at least some do. And I am also able to catch students who are in serious need of help (e.g. those students who really lack certain basic skills of composition) and suggest to them that they go to a lab. Perhaps one way to address these latter students is to receive a draft from each student and suggest that those who are in need of going to a lab do so. But I still don’t know how to ensure that each student gets at least some useful feedback without my giving it to them.
A very helpful article on peer reviewing of student papers is Stanley Werne, “Taking Rough Drafts Seriously,” Teaching Philosophy 16:1 (March 1993), pp. 47-57. It has a number of practical suggestions for successful peer reviewing in a philosophy class. I read this article a couple of years ago, and it inspired me to attempt this in a medical ethics class, and the experiment proved pretty successful. I recommend Werne’s article to anyone who wants to try peer reviewing.
I hope unsolicited comments are ok for everybody here…
I am but a silly grad student who has done just a bit of student teaching, but it’s been my experience so far that the best (general) way to improve students’ writing is to simply get them to write more. Turns out that while comments do help, the process of writing alone seems to help improve writing substantially. This works particularly well in the aforementioned lower division/intro courses, I think.
I’ve done some in-class peer review, with pretty good results. Groups of three reading each others’ short papers and then comenting orally on them seems to work better than written peer review for me, simply because people are less likely to spout complete, utter fluff while surrounded by peers. They still will, of course, just not as often.
As a student I experienced a similar sort task but it was done a bit differently to avoid the “Awesome” problem…
We read a metaphysics paper by Lewis I believe, and half the class was given the task of summarizing the work within the boundaries of 2 pages. The other half of the class was tasked with evaluating the 2 page summary. Everything was nameless, so that no one knew who critiqued thei paper, nor did anyone know whose paper they were critiquing… The grades were given based on each persons work.
Pedagologically the Professor was able to see what each student learned about the Lewis text regardless of which group they were in (summarizers or critiquers), so the playing field is equal, do to speak. On the next article we changed groups so that we did the opposite of what we did before.
I don’t think you said how big your class is, but if your class is not that big (not much bigger than 20, tops), it is helpful for them to first all read one student’s paper and then talk about it all in a group, with you leading the discussion. Basically, you guide them through the peer-review process, showing them what kind of questions to ask, how to (kindly) criticize their peers’ papers (and their own), etc. Ask them what they liked about so and so’s paper, what they didn’t, what was unclear, what it the thesis seemed to be, whether it was adequately supported, whether objections were considered and responded to, etc.
Doing this kind of thing as a huge group activity helps them see what you are looking for since, typically, they have no clue (and they shouldn’t, since they’ve typically never done anything like this).
So that’s my suggestion: start these kinds of things as rather large group activities and then let them loose in smaller groups.
One way that eliminates the problem of apathetic peer editors is to hold them accountable for the editing skills. I give a questionaire to every peer editor outlining the information that should be considered and critiqued, and then I collect the peer edit questionaire with both the rough and final copies of the essays. I grade the peer editor on the advice and feedback that was given to the writer. Hope this is a helpful suggestion.