Grad student here. I was chosen to referee a paper for a top field journal (sorry!. I presented at some conference so that's how I am guessing I got onto the list.
Is there a standard guide as to how to do this. Length, what to cover, how to criticize, etc. I asked my advisor for help and she basically told me that I'm on my own.
Thanks for all the help.
If this is a very good paper, recommend R&R.
If this is a very poor paper, recommend rejection.
If this is a marginal paper. Google the authors. If there are low-ranked monkeys. Try to find some technical errors, then recommend rejection.Or simply said, nothing is novel.If the authors could be your futher connections, try to find some technical errors, and suggest them to modify it. More importantly, find an appropriate way to suggest them to cite your paper. By doing this, you implicitly tell them "I am the referee, I do you a favor! remember me!".
Well, I reviewed 13 and rejected 11. There is selection: when you are an AP editors tend to give you clear decisions.
I always do my job seriously, take a lot of time to read the paper carefully and write very detailed comments. For a first submission, I've never written less than four pages. And I always let some time pass between the first draft of my report and the final version: I want the idea and the approach taken to sink in, to be as impartial as possible, and for me it takes some time.
But I think you have a point: as AP sometimes we lack the experience to see through the contribution and understand if it matches the level of the journal.
For example, I had a rejection of a minor paper I wrote a few months ago. One referee was clearly a senior, who pointed out in a beautiful way the flaws, and suggested a minor revision. The other was clearly a junior, who got mad for a relatively secondary point and thrashed the paper with few quite rude sentences.
I know one AP, he told me that he has reviewed 17 papers so far. He recommended rejection for all of them!
Copy paste from a post (not mine) I saved from sometime ago. Here it is.
*********
I am an editor at a good journal. (So I have an interest in explaining what I'd like from a referee report!)
Here are my thoughts:
1 - How long to you take per review? How does it change with the ranking of the journal?
When you are fairly new at refereeing, plan to spend at least a half-day on a paper. You may end up spending a full day on some, less on others. But providing a good referee report, especially for a good paper at a good journal takes time.
As you gain experience, you'll be able to do the review faster.
2 - Do you bother listing typos? Is that a good thing to do? Even for terribly written papers (UHF of typos).
Listing some typos shows that you read the paper carefully. But your job is not to proofread the paper. It's fine to say that there are a lot of typos (or grammar mistakes, etc.), give a few examples, and note that the paper should be thoroughly proofread. If it is clear that the paper was written by a non-native English speaker, it is fine to suggest that the author hire someone to improve the language use.
My view (based on writing, being a referee, an dissertation supervisor, and editor) is that empirical papers that have a lot of typos also have a lot of programming errors. People who write sloppily probably code sloppily also.
3 - Any good practical tip?
Yes.
A. Your job as a referee is to evaluate the contribution of the paper. Your role is not like that of a dissertation adviser. A good referee report has the following components:
A1. A *short* summary of the paper. This lets the author and editor know you understood what the author was trying to say.
A2. A concise statement of the contribution of the paper to the literature. I read referee reports that list a bunch of questions and suggested changes. These are fine, especially if I'm going to accept the paper. But what I really want to know is whether the paper makes a contribution to the field. A good paper clearly explains the contribution, which makes the job of the referee a little easier. But the job of the referee is to give me an independent assessment of the contribution.
A3. Then go on to list specific comments on the paper or suggestions to improve it. If the paper receives a revise and resubmit, the editor will likely ask the author to give you a point-by-point response to your questions/suggestions. Keep this in mind when you write the report. If you'd like to see something specific, be clear about what you want to see.
A4. Finally, you will write a note to the editor that will not be seen by the author. This note should say why you think the paper should be accepted, rejected, etc. If you aren't familiar with the standards of the journal (which is a pretty common situation), you point out other journals that you think this paper is appropriate for. Frankly, don't fret too much about whether the paper should be accepted or rejected. It is very hard to figure these things out when you are young. It is far better to write a good review.
Editors are not bound by your accept/reject recommendation. Note also that just because an editor accepts a paper that you recommend rejection, or rejects a paper that you recommend be accepted, doesn't mean the editor "ignored" you. The editor knows the pool of submitted papers; you don't. The editor may rely heavily on your assessment of the paper even if he/she doesn't follow your suggested decision. I receive a number of referee reports that recommend acceptance but it is clear from the referee report that the paper is not very good.
I want to emphasize that saying that a paper is competently done does not imply that it should be accepted. A paper being competently done should be a requirement at any journal. What differentiates the best journals from the rest is the importance of the contribut...See full post
Just to be serious, editors far prefer being told you cannot do it than for you to say yes and not do it or just not reply, unless you are a bigger cheese than they are, which most readers of this place will not be, and certainly not junior whippersnappers.
A generalization is also that junior people are more expected to do the checking on details of proofs or econometrics, while it is the more senior referees who are expected to make the more serious evaluative comments on where it fits in with existing literature and all that, although a junior person should offer a judgment anyway.
Just looked at the link and below where they have
"Customers Who Bought This Item Also Bought.."
There was Body Shampoo:
I found this book helpful:
http://www.amazon.com/Guide-Young-Economist-William-Thomson/dp/0262700794