Menu:
Peer Reviewer: Friend or Foe?
by Robert P. HolleyI've been a peer reviewer for so long that I can't remember when I started. I've reviewed manuscripts for around ten journals, including some that are the most highly rated in their areas as well as several respectable second-tier publications. I've looked at work from both very senior scholars and beginning authors. I believe that I get more manuscripts than the average reviewer because I send my comments in very quickly, often in less than a week, and because editors tell me that authors find my comments helpful in improving their publications. E-mail and journal web sites have greatly simplified the process from the earlier days of mailing manuscripts back and forth.
General Comments on Reviewing
I approach each article wanting to like it. I normally don't care all that much about small problems in phrasing and punctuation, since I know that the journal's editor will handle these changes. Instead, I pay the most attention to the article's overall subject matter and methodology. Do the authors have something worth saying and say it well? Are the conclusions adequate and justified by the evidence that the authors present? Does the article have a clear structure? I tell editors up front that I have a humanities background and won't be able to comment on the statistical analysis, adding that they should send the article to at least one other reviewer with those qualifications.
I read each paper quickly to get the overall feel for its contents, and make some preliminary notes. If I feel that the article is not suitable for publication, I stop there and return it with my reasons to the editor. This has happened only about three times in all my years of peer reviewing. Otherwise, I come back for a more careful second reading and take additional notes, including selected quotes that will help me make my points. I try to keep my evaluations short and concise, but I do some of my best thinking as I write - so they often turn out longer than expected. I normally put my suggestions in priority order, telling editors which changes, in my opinion, are critical, and which would make the article better, but are not "deal stoppers." As a writer myself, I am especially cautious about suggesting any major additional research, though I have been known to ask authors to do some additional literature searching to place their work in the context of prior publications.
I sometimes compare the final journal article with my comments and find it rewarding to see that many authors have made most, if not all, of my suggested changes. While I know that there are criticisms of the peer reviewing process, I believe that I have made a difference in the articles that I've reviewed.
Specific Suggestions
- Look at your article from the reader's perspective
One of my most frequent comments is that authors should not assume that the reader has the same knowledge as they do when writing the article. This can be as simple as giving the size of their library, to provide needed context, or as telling the reader right away that their subject is "federal United States" government documents - since government documents can also be state or international. Authors should consider the journal's readership. I recently reviewed an article about a library in Africa, where I kept interpreting the content in light of my American experiences before realizing that I was probably making some incorrect assumptions. In this case, the author should have helped me better understand the specifically African context, since most readers of the journal would, like me, be unfamiliar with African culture.
- Put conclusions within an appropriate context
One of the more common mistakes authors make is to present a case study and draw more general conclusions than are warranted, since specific local conditions may have influenced the results. On the other hand, if a case study deals with issues known to be of general interest to the professional community, authors should take care to discuss these aspects. To give an example, I reviewed an article on a digitization project where the library was watermarking all their images. Nothing, however, was said about possible future efforts to recoup the project's costs by licensing these images, a "hot topic" in this area and the usual reason for watermarking. Another danger is to treat a subject as brand new, when it has been around for a long time. I once told an author that a "new found problem" dated back to the 18th century, and gave a specific example.
- Consider whether there is there another explanation for the
results, or another possible causal relation
Authors should try to step away from their results to make sure that there isn't an alternate explanation for what happened. Causality is difficult to prove. A good author looks at other possibilities and deals with them in the article. On a related issue, in a recent article, an author described "Project A" and "Project B" as if the two were intrinsically linked. In fact, the library could have initiated Project B, albeit less successfully, without ever having undertaken Project A. I also check for internal inconsistencies, where two parts of the article are seemingly contradictory.
- Ask yourself what has been left out
As a peer reviewer, I always look for what has been left out - either intentionally or unintentionally. I strongly believe that the best way to deal with negative evidence is to present it with an explanation of why the author's point is still valid. Once again, in reviewing an otherwise excellent article for a prestigious journal, I felt as if the author were hiding a very minor piece of data (that, at worst, would have undermined the ultimate conclusions only slightly). If authors are unsure about an additional fact that they don't have the time or methodology to evaluate, they can always correctly state that this is an area for further investigation.
- Pay particular attention to the abstract and the first and last
paragraphs
Many potential readers will see only the abstract of your article, or will start with it to see if they wish to read the rest. The next hurdle involves generating enough interest in the introductory and concluding paragraphs to hook the reader on the complete article. I always pay special attention to these parts of the article and frequently have suggestions for improvements. In one case, I suggested that an author, who was proposing new terminology, use older terminology in the abstract, since readers would most likely be confused by the new terms.
The peer reviewer can be the author's best friend in creating a much stronger article than the initial submission, even if the changes can sometimes seem picky and unimportant. At its best, the peer review process finds errors, methodological problems, and inconsistencies while authors can still correct them - a much better alternative than learning about them in a letter to the editor.
Bob Holley is professor, Library & Information Science Program, Wayne State University, Detroit, MI, after having been an academic librarian for 20+ years. His current teaching and research interests include collection development, the out-of-print book market, popular culture, and intellectual freedom.