After the success of the last year peer review workshop, beginning of this year Martijn Wackers, teacher of a new group of students following the Collaborative Science for Biomedical Breakthroughs (Leiden Delft Erasmus) Minor, organized together with TU Delft OPEN Publishing a new peer review training session. We designed the workshop for the students to experience first-hand the value of peer review by working on their own drafts of their first research papers.
The workshop
Three groups of students working each on a scientific paper, were asked to review another group’s paper. Therefore, the students participated in the workshop in both roles, as authors and reviewers.
The workshop was organized in three parts:
- An introduction to peer-review
- Peer-review in practice and
- Feedback
In An introduction to peer-review we discussed with the students the value of peer-review, presented a short history, and the types of peer-review with emphasis on the Open Peer-Review. We explored further the open review with and without open identity and the preprints, and the ethical implications of maintaining the high-quality standard required for a rigorous scientific publishing. As examples of journals with open peer-review, they could see the TU Delft OPEN Publishing JOAS, The Evolving Scholar/ BMD 2023 Conference community, and HESS, published by Copernicus, each with their own publishing model.
During the second part, Peer-review in practice, the students had to read the assigned paper and write a constructive review. We asked them to concentrate on a general understanding of the paper and on the identification of major and minor issues.
The whole assignment set-up was developed by Martijn in Brightspace, where the students could use Feedback Fruits tool for writing their individual review.
In this step we used instructions and tools developed by PREREVIEW:
These materials were sent to students before the workshop to study them. For practical reasons, we made a short introduction, and a scheme of the important steps a reviewer needs to take to make a good review.
With the printout, we offered the students a practical structured approach to understand the article, with examples of right questions to ask for every section of the paper. Moreover, to help them formulate their review, we gave them a list of appropriate sentence starters.
During the Feedback, after writing their individual review, we asked the students to discuss their findings and collaborate within their group to write one final review as one narrative. After the discussion, all the groups presented their findings and had a constructive reviewer-author-reviewer discussion.
The Results
The students had very relevant and sharp feedback to their colleagues. Reviewing someone else’s work helped them to self-reflect over their own approach and structure of the paper.
As authors they learned to:
- avoid rewrite what others did to reduce the length of the paper, reduce one page to one line of explanation with a reference to existing work.
- keep the focus on what one wants to communicate in the paper and try not to lose the readers with too much detail
- add definitions to the reader when introducing new concepts
- add the extended methods and protocols at the end and not in the middle of the article
As reviewers,
- they recognized the value of the review in an early stage, which pointed to the value of preprints.
- the students appreciated the open identity model, realizing that they want to be able to trust the one who reviews their work; on the other hand, they would be happy to know if multiple (eventually unprofessional) reviews are made by the same person.
- one student was glad to finally understand the difference between general feedback and a professional peer-review and the responsibility that comes with it.
- another student appreciated learning from the print-out how to do a structured review, per sections of paper, so that she can be thorough in her review.
Feedback on the workshop
This workshop was built upon the evaluations we picked up from the last year students:
- we adapted the assignment, from one paper section to the whole paper.
- we adapted the approach: the reading & writing exercise, instead of fixed time windows for each, they were merged in one long exercise. This allowed the students to manage their own time of reading and writing, as they found it fit.
- we steered the review towards the most important points of critical thinking and improving the work, leaving out the language and the phrasing analysis.
As a result, the students appreciated the design of the workshop. The times allocated were just right, the introduction was useful, and the reading & writing time (75 min) was very good, with the exceptions of two students taking their review work very seriously and in great detail and who needed more time.
The students admitted that they didn’t have time to read the help materials provided before the workshop. Therefore, they were happy with the hands-on approach we had, with the printout and the presentation of the reviewer guide summary.
Last, all the students appreciated the indirect help of the workshop, that pushed them to write the first draft paper. Moreover, getting a review from their peers early on their drafts, helped them write a better scientific paper for the end of their research assignment.
Looking forward
Reviewing is a skill that needs to be acquired in academia for a rigorous scientific publishing, in order to maintain the credibility and the integrity of the future scholar work. The results of this workshop prove that healthy peer review practices can be learned early on by every new generation of students. Would a teacher be interested in including hands-on peer review in the curriculum of their students, let us know.