I am currently reviewing one empirical paper and some of the results look fishy to me. Should I send the Editor an email requesting author's code and data? Have you done so before?
Do you ask for author's code as a referee?
-
Flip side: I've always included a code archive in the submission. As far as I can tell nobody ever looks at it.
This sounds like a bad idea. Even if you haven't made any mistakes letting the refs looks through all your do files and data seems like it could open a Pandora's box of new criticisms. This surely outweighs any reputational benefits of "transparency."
-
Flip side: I've always included a code archive in the submission. As far as I can tell nobody ever looks at it.
This sounds like a bad idea. Even if you haven't made any mistakes letting the refs looks through all your do files and data seems like it could open a Pandora's box of new criticisms. This surely outweighs any reputational benefits of "transparency."
Maybe, but three papers now and nobody has mentioned it a single time. It could be that I work in R.
-
The thing is they criticize without fully understanding reports. I can't find a person who knows what is stationarity.
Don’t be a clown. Referees are not supposed to be code, or data, police. You will be viewed as unprofessional by the editor.
And that's why 4b1f is 100% right: academia peer reviews are less rigorous than econ consulting criticisms of opposing reports.
-
Don’t be a clown. Referees are not supposed to be code, or data, police. You will be viewed as unprofessional by the editor.
And that's why 4b1f is 100% right: academia peer reviews are less rigorous than econ consulting criticisms of opposing reports.
You say this like anyone should care. Why the hell should they? Literally billions are at stake in econ consulting cases. Who in their right mind thinks the consequences of the latest in fetalnomics measures up to that?
-
Definitely do this! It sounds like some of those who recommend against might have submitted something based on code they know to be fishy...
In any theory paper you’d be expected to check the proofs. So of course, if there seems to be something fishy in an empirical paper, you should check the code!
-
Of course you should ask for the code and check it. Anyone suggesting otherwise is a sad excuse for a social 'scientist'. Of course everyone needs to worry about their career trajectory. But how much integrity are you lot willing to sacrifice for strategic purposes? Do you have any respect whatsoever for the work that you do?