Developing doctoral students’ critical writing skills through peer assessment and review

on Doctoral Writing SIG:  https://doctoralwriting.wordpress.com/2020/11/13/developing-doctoral-students-critical-writing-skills-through-peer-assessment-and-review/

Our guest blogger this week is Dr Joan Woodhouse, Associate Professor of Education at the University of Leicester UK. She has been working with doctoral students for approximately thirteen years and has a keen interest in developing early career researchers’ academic writing skills. Her other research interests include teachers’ lives and careers, and in particular, women teachers’ lives and career experiences. She is currently engaged in collaborative research into the experiences of student teachers who are mothers, with a view to considering how providers of Initial Teacher Education might better support this group of women, who she sees as the ‘invisible statistic’ in the equality monitoring data.

By Joan Woodhouse

As Programme Director for the Doctorate of Education (EdD), I have worked with numerous full-time teachers and school leaders who combine their professional work with part-time study. The EdD students are typically mid- to late-career teachers, often occupying fairly senior positions in school. As doctoral students they can be rather isolated from other, often younger, full-time and campus-based postgraduates. Their main point of contact with the University is with their supervisor. So, it can be, at best, challenging for this group of students to feel a sense of belonging to the academic and research community. Many of the EdD students are also returning to academic study after a significant gap and can feel uncertain in navigating the unwritten rules and culture of academia. They are expected to read and write critically, often without any explicit guidance on what that means. They are given feedback on their work in which the weaknesses in their arguments are highlighted, their poor expression exposed and the gaps in their knowledge indicated. It can be hard to take the critical feedback: affective barriers can impede students’ ability to assimilate and act on the feedback, yet assimilation and action are vital if they are to progress. Learning to respond to feedback in ways that are pragmatic rather than emotional is a big part of developing resilience as a researcher. I have come to realise that as supervisors and tutors we need strategies to foster this resilience in our students if they are to become the critical thinkers, readers and writers we need them to be to succeed at doctoral level.

This post is about a project in which a colleague and I inducted doctoral students into peer review by (i) involving them in peer assessment activities and (ii) supporting them to set up and run a student journal. Our aim was to facilitate the development of students’ critical writing skills, by engaging them in giving and receiving critical feedback (both of which they were fairly anxious about at the start).

We interviewed the students at various points of the project, to ascertain what they perceived to have been the impact of the experience on their learning and their development as critical writers. The third and final phase of the interviews took place after the student-led journal had been running for just under three years, by which time the students reported feeling much more confident about providing critical feedback to others, and more open to accepting critical feedback from others. Moreover, by reviewing others’ work they had learned to look more critically at their own writing. Their understanding of criticality was considerably enhanced and there was a sense in which they had become inducted into the culture of academia.

Lots of students find critical writing difficult. Supervisory feedback frequently advises students to ‘be more critical’, but it is not always made clear to students what ‘being more critical’ actually means. As supervisors we need to be more explicit about what we mean by criticality, and to find ways to scaffold students’ learning. I describe here how we inducted students into the process of peer review by devising peer assessment activities and challenging them to establish their own journal.

The journal project

Ten part-time doctoral students (all full-time teachers) volunteered to be involved in the project. Part-time students can feel very isolated, as for most of the year they are away from the University and only have contact with other students at the annual summer school. Although some of them had set up their own support group prior to our project, they had themselves remarked to us that they were ‘too polite’ to provide each other with any real critical feedback on their work. Whilst they liked the contact with other students, this ‘tea and sympathy’ group did little to develop their thinking and writing skills and did not provide a sufficiently strong sense of being engaged in a research community. So, we wanted to give students a sense of purpose, and of meaningful engagement in a research community. Aiming to foster their independence, we sought to scaffold and facilitate their learning through what would become a student-led and student-sustained writing and peer reviewing community.

There were two phases to our intervention. We began by running a three-day critical writing residential programme. Prior to the residential programme, participants were contacted and asked to email a draft piece of work (up to 2,000 words) on which they would like to receive some formative feedback. Each draft was then forwarded to two other participants, who were asked to read and comment on the work. A pro-forma was sent out to help participants to structure the feedback. Prompts on the pro-forma included:

• • • Clarity of language and expression
    • Critical engagement
    • Development of argument
    • Three strengths of the work
    • Three areas to improve

Day one

On day one, we scheduled a ‘roundtable’ meeting, during which each participant gave and received two sets of oral feedback on the draft work. The feedback was shared publicly with the group.

Following the roundtable session, participants were asked to reflect on the feedback they had received and to identify a small number of issues to work on in their own writing, enabling them to draw up personal action plans.

Day two

We ran three workshops on day two. The first, ‘Positive Criticality’, focused on what is meant by ‘criticality’. Emphasis was placed on criticality as synthesis of ideas, discouraging the view of criticality as a deconstructive process focused exclusively on ‘proving’ arguments to be ‘wrong’. We stressed that critical writing involves communication rather than just deconstruction. It entails combining ideas in novel ways to offer new insights. We stressed that developing ideas is a slow process that happens gradually and takes time. The need to read, write, draft and re-draft as a part of this process was discussed.

In the second workshop, ‘Developing Clarity and Criticality’, the focus shifted to the form as well as the content of writing. Students were asked to look at a journal article and asked to comment on the following:

• • • Structure
    • Use of language
    • Use of literature
    • Clarity of message
    • Use of data and quotes from participants
    • Use of evidence to substantiate the conclusions
    • Any other observations about what makes this article effective?
    • Are there ways the article could be improved? What advice or feedback would you give to the author if so?

The discussion during the session focused on the elements of good, critical writing with reference to concrete examples in the article. These examples included a focus on the form and the content of the writing as well as the development of the argument. The workshop culminated in another opportunity to engage in peer assessment, this time explicitly focused on the form of the writing rather than argumentation, use of evidence and so on. Students were asked to look again at their colleagues’ draft work, now with the following brief:

On your partner’s draft work, mark:

• • • Sentences and sections that are clear and effective. Comment about why they are effective.
    • Any superfluous words/phrases s/he could cut.
    • Any jargon s/he could re-phrase in simple language or define more clearly.
    • Any ungrammatical sentences.
    • Any sentences with multiple sub-clauses or sets of brackets that could be re-written as shorter, clearer sentences.
    • Any paragraphs without a key point or message.
    • Any sections that don’t match the sub-heading under which they appear.
    • Any sections in which the author is listing readings rather than developing an argument.

In the third workshop of the day, we prepared students for the next stage of the intervention, when they were to form an editorial board and set up an online journal. Our rationale for the second stage of the intervention was that, through their involvement in an editorial board of an online journal, students would be required to apply their peer assessment and critical writing skills in a real context, and become inducted into the culture of peer review and publication that is so much a part of academic life.

The participants took the lead in establishing the board, the policy and the strategic plan for the journal. Roles were negotiated: an editor and assistant editor were appointed. The rest of the residential programme was given over to the participants to work together, ending with a presentation from students in which they explained their plans for the journal, with details of dates and deadlines.

The editorial board devised and ran an impressive, online, open access, peer-reviewed student journal. They produced two issues per year for over three years in which research papers, opinion pieces and book reviews written by doctoral students were published. All members of the editorial board undertook peer review of submissions, which were revised and re-submitted by authors, as with any reputable academic journal.

This project could be adapted to suit other disciplines. From our experience, it would seem that up to around fifteen students might conceivably be involved in the editorial board, although a wider group of students might feasibly be involved as peer reviewers if they were offered ‘training’ in peer review. We strongly recommend this process. Our expectations of the students were high, yet we were stunned by their response to the challenge, and the learning they gained from it.

For a full report on our project please see:

Woodhouse, J. & Wood, P. (2020) Creating dialogic spaces: developing doctoral students’ critical writing skills through peer assessment. Studies in Higher Education. DOI  https://doi.org/10.1080/03075079.2020.1779686