You can always tweak your micro model to tell a fitting story. As someone wrote below the post:
''There is a lot of elasticity in what that model can say, a whole universe of tweaks.''
Also, see the comments below the post about the difference between p-hacking (in the sense that you actually run 100 tests) and the garden of forking paths (i.e. potential comparisons). See e.g. this comments by Daniel Lakeland below the post:
''Right, there’s a difference in *algorithm* between “try every possibility and take the one you like best” and “make a intelligence-directed random walk and at each step only backtrack if it works out very badly” which can look quite linear with maybe only a couple backtracks.
So if you get from the root of a tree to a leaf with only say 2 backtracking choices that you explicitly mention in your paper, it can feel like you just “got to the answer”. But when it comes to interpreting your p value, the potential branch points were so numerous that the “try every possibility” method might easily have included millions or trillions of potential options.
It’s like saying that chess is trivial because you won a particular game. There were trillions of possible other games you didn’t play in which you might well have lost. The “directed thought” you put into your chess moves is what made you win, not that “the answer to chess was always this sequence of moves”''
Finally, while you might be right that ''their arbitrary specification will either be broadened or challenged by subsequent work'', it might just as well be replaced with other nonsense. Especially if the truth is not publishable. Also, the specification is only a small part of the problem, there are thousands of potential choices that can be made. Economists may publish an Online appendix with 30 robustness checks, but what is really needed is some type of multiverse analysis.I don't think we have a problem of p-hacking in economics. Yes, individual authors p-hack. However, eventually, their significant result will have to fit into the standard micro framework, their arbitrary specification will either be broadened or challenged by subsequent work. It's micro theory that save economics. Psychology and sociology have no such discipline.
I'm still not getting your point, which also seems quite hypothetical and requiring many helpful assumptions. Can you name one established result that you question because you think it was p-hacked? My point is, if a result is p-hacked, it won't make sense. Some time later, someone will want to challenge it.