I think that this story illustrates many of the things that are wrong in our profession:
https://twitter.com/ClaudiaTroccol1/status/1410630154460844032
Comment, editor letter, and referee reports here:
Economic Journal rejects a comment against the opinion of the referees
-
Based!
Based on what?
Check the editor letter: the editor argues that EJ does not accept any comments, but they were not aware of that when they sent the paper to referees. Not clear how they expect readers to find out that the results of a published paper are wrong.
-
This is exactly what I am afraid of political science, too. I made a replication for a paper in top field and the main results go away when you make the correct, mathematical assumption about multinomial models. I was so afraid that the authors, editors or others will think negative of me or reject my replication, I did not even submit it. Are there any replications that get published as comments?
-
Based!
Based on what?
Check the editor letter: the editor argues that EJ does not accept any comments, but they were not aware of that when they sent the paper to referees. Not clear how they expect readers to find out that the results of a published paper are wrong.
As usual the best source for knowing whether the results of a paper are wrong is EJMR. This says a lot about our profession.
-
If you guys read the comment, the mistake in the original paper is particularly revealing. Results are only significant when the authors use one of the 5 plausible values of PISA, they disappear if you use any of the other 4 values, or if you use the appropriate procedure (average of the 5). Note that each of these 5 values represents a random draw from a posterior distribution.
-
I don't see the big problem here. She can just submit somewhere else.
A comment on a paper must be published in the same journal, if you try submitting it elsewhere it will be rejected. You could try and turn it into a separate paper but the contribution is not substantial enough for that, which is why its a comment in the first place.
-
Original authors fake their way to publication. Journal protects them, will do NOTHING to warn the 'profession' of the bogus results. I learnt that's most journals' SOP. I don't believe anything from an econ journal anymore until I've replicated it myself - hence, there isn't a whole lot of empirical results I believe in...
-
I don't see the big problem here. She can just submit somewhere else.
A comment on a paper must be published in the same journal, if you try submitting it elsewhere it will be rejected. You could try and turn it into a separate paper but the contribution is not substantial enough for that, which is why its a comment in the first place.
(1) Personally I find this is pettifoggery. Slightly rewrite the paper and submit elsewhere. The point is made wherever you publish, especially if the evidence against the original is that strong.
(2) This has happened at the EJ a couple of times before, see below (also a good example of 1).
Comment
https://www.sciencedirect.com/science/article/pii/S0165176512003229#br000005Original
https://academic.oup.com/ej/article/121/550/F59/5079707?login=true