1. 1. Posted by Thom Brooks | July 25, 2008 11:11 am

   This is an excellent idea, Matthew. It will be interesting to view the poll results after a few weeks. Putting my editor’s hat on, the biggest problem I’ve faced is when you have the right referees for a piece who are willing to review, but stumbling blocks continue to arise. Thus, when ‘can you send me a review next week?’ is met with ‘no problem’ and then this deadline is passed, things become complicated. Of course, it is simply true that referees work for free and for the good of the profession, with plenty of constraints on their time. While I believe it is important to review submissions in a timely fashion — (the JMP normally reviews 80% of submissions in two months or less, although this number is 98% thus far in 2008) — I am sympathetic with reviewers who need more time. That said, I think the six month and beyond barrier is unacceptable.

2. 2. Posted by Doug Portmore | July 25, 2008 1:29 pm

   I think that the more important question to ask is whether you’ve ever taken longer than you agreed to. I have taken up to eight weeks to review a paper, but when I’ve been asked to do it in a shorter period of time, I’ve always met the deadline or, if there was a possibility that I couldn’t, I’d decline the initial request to review. When I get a request to review a paper, then, it always comes with some remark such as “and we would need a decision within n weeks. If you cannot meet this deadline, please suggest someone else who would be an appropriate reviewer.” What’s surprising to me is that journals vary in the time that they give me: anywhere from 6 weeks to 12 weeks. Sometimes they give me less time, as there was some initial hiccup with the first reviewer. Also, it’s surprises me that sometimes I get requests to review by post. This is bad, because I don’t check my campus mail that often, especially during the summer. So although I always respond quickly to requests I receive by email, I don’t always respond quickly to requests I receive by post.

3. 3. Posted by Simon Kirchin | July 25, 2008 2:44 pm

   I agree with what Thom and Doug say. Here’s another point worth making. Putting my editor’s hat on (Ethical Theory and Moral Practice), I can say that the single biggest problem at present is not so much tardy reviewers, but rather the time it is taking to find reviewers. Once the editorial team have read papers and made a decision to send out to two referees. (which takes around two days to a week), we have to find those refs. Although we always end up with decent reviewers – and I hasten to emphasize that for any potential authors out there – I hardly ever get my first two choices. More often than not I have to ask about five people to get two people to say ‘yes’. Occasionally I have to work through a list of about ten people. Even carrying out all our activity electronically, finding two people to review can be time-consuming and add to the length it takes to reach a decision. The length of time increases if people simply don’t bother to reply even when they are going to decline. (We typically allow people ten days to respond to requests at ETMP, with reminders, and then they are automatically deselected.) I can’t imagine that ETMP is the only journal in the same boat. Indeed, with my referee’s hat on, over the past month I have been asked to review five papers, and in an average year I get around 20 requests. I can’t imagine I am first choice for more than a quarter of these.

   This topic comes up in bar discussions at conferences, and on blogs, every so often. Doug’s suggestion is a good one, as is his journal wiki (whose URL I can’t locate right now). Sharing information on journals’ practices has to be a healthy thing. But to be blunt, as far as I can see we have an honour system that it is too easy to opt out of, or not get involved in in the first place. The way to resolve things once and for all is to have a direct incentive to review for a decent number of journals and do so in a timely and thorough fashion. The obvious incentive would be to tie this to being allowed to have your papers considered by the journal(s) in question. But this creates a clear and large conflict of interest, since it would be too tempting to reject a paper so as to allow for a greater possibility of having your paper accepted (which it wouldn’t be, of course). Any other direct incentives people can think of? Do people really care about having a journal discount? (Writing this has made me depressed. I need cheering up. Now, where did I put that paper I promised to review?….)

4. 4. Posted by S. Matthew Liao | July 25, 2008 3:47 pm

   Doug says:

   I have taken up to eight weeks to review a paper, but when I’ve been asked to do it in a shorter period of time, I’ve always met the deadline or, if there was a possibility that I couldn’t, I’d decline the initial request to review.

   What Doug says here is exactly why I think that it might be a good idea for journal editors explicitly to ask potential referees if they can review an article within a certain period of time. This is a bit of an arm-chair speculation, but it seems that people may be more likely to deliver their verdicts on time if they have explicitly agreed to a particular time frame. I think also that it may be worthwhile standardizing some kind of reasonable time frame for the refereeing process and that the profession may have an interest facilitating the adoption of such a time frame.

   Simon and Thom, I’d be quite interested to learn how long on the average it takes to find two reviewers in your respective experiences as editors.

5. 5. Posted by S. Matthew Liao | July 25, 2008 5:34 pm

   By the way, Doug’s very useful Philosophy Journal Information wiki can be found at here. I’ve also added it to the Philosophy Resources Blogroll here.

6. 6. Posted by Thom Brooks | July 29, 2008 11:28 am

   Simon is absolutely right — except, of course, he’s always top of my list for reviewers for the JMP! Part of the trick is coming up with a list of names, but then they must be found, and then one hopes they will agree. I know in political science that the American Political Science Review normally asks authors to suggest four or five names: a laudable idea, except I’d have some concerns.

   The wiki seems fine, too– I just hope it includes the JMP soon!

7. 7. Posted by Neil Levy | July 30, 2008 6:57 am

   I don’t see why we give referees 3 months. 2 weeks is plenty. If you can’t do it within that time, turn it down. But I also think that by submitting to a journal you commit yourself to reviewing two papers within the next 12 months.

   I’m not participating in the poll, on the grounds that I have never taken 1 month to referee a paper.

8. 8. Posted by Mark van Roojen | August 1, 2008 4:43 am

   Doug is right. If people mostly did what editors ask, much of our problem would go away. I’ve been given anywhere from 3 to 12 weeks to referee. Most ask for a report in 6 to 8 weeks, and I recall none with no deadline conveyed in one way or another. I’ve refereed a lot so I think I actually have a decent sample. If all Journals went to the 6 to 8 week model, if reviewers met deadlines, and if editors could find referees in under 4 weeks, we’d be pretty close to the 3 month response time that I was led to believe we should expect from journals in the profession when I was starting out. Even those journals which give reviewers 3 months would not miss it by much.

   I also think Neil overgeneralizes from his own work habits and situation. Two weeks may well be plenty for him, but I’d rarely accept an assignment if I was told I had to do it in that time. I suspect there is a spectrum of times that would work for different philosophers. I’m in the month to 2 month range, depending on my other commitments. But then I’ve sometimes had to buy a book and read parts to do a decent job on a review I’ve had to write. Since I think that 3 months for a well-thought out set of comments from a journal is not unreasonable, I don’t think there’s anything all that wrong with that. And I think if we cut out those unwilling to do a review in two weeks we’d have half as many reviewers. That would likely raise the average response time in itself.

9. 9. Posted by Neil Levy | August 1, 2008 4:50 am

   Mark, it is possible that I do over-generalize from my own situation. David Chalmers noted on Leiter that he has the same habit, but Dave and I have something relevant in common: research jobs. Still, I do think that if referees were required to turn around manuscripts more quickly, they would: given 8 weeks, the temptation is to place a low priority on the job. You said that editors should find referees within 4 weeks. Here, too, I have relevant experience: I am an editor as well. Of course, people knock you back and that can drag out the process, but that aside there is no reason for editors to take so long to find referees. It should be done within a few days.

10. 10. Posted by S. Matthew Liao | August 1, 2008 2:33 pm

    Just a quick clarification. I know that most journals do ask for a report within a certain timeframe. But as far as I am aware, most journals do not explicitly ask potential referees to decline a request to referee, if the referees foresee that they can’t do it within the allotted the timeframe. The consequence of not asking explicitly means in practice that the timeframe given by the journals is treated more like a suggestion rather than an expectation. My thought is that making this timeframe an expectation rather than merely a suggestion would benefit us all.

11. 11. Posted by Mark van Roojen | August 1, 2008 5:12 pm

    Neil, I saw Dave’s comment on Leiter as well. Since I know him to be impressively quick at everything he does I’m not surprised, but you are probably also right that the nature of an appointment and the teaching duties it involves may be relevant. I just wanted to emphasize the range of people’s habits and that for some of us we’re generally overcommitted in the near term, even though we referee and largely meet deadlines.

    Your second comment actually reinforces what I had wanted to say. If referees can be found more quickly than I conjectured, it would be possible to get papers back even sooner than I’d said when I was making my point, so long as people actually met deadlines as Doug suggested they should. I wonder if all journals find it so easy to get referees in that time frame, but I’d be happy to hear that the answer was yes.

12. 12. Posted by Nicolas Espinoza | October 6, 2008 9:25 am

    Here is a way of expediting the review process: When someone submits a paper to a journal they will not get a response from that journal until they have reviewed someone elses paper for the same journal or publishing house. The more papers you review the faster your paper will be handled/published. Or something along those lines.

Post a comment