Close

Complete Captcha

• 1 year - 94 posts - Latest - RSS
• Thread: 149 Goods vs 44 No Goods

1. Economist
   4d19
   

   That is useful, but might not quite get at the opportunity cost angle. I chose to be a part of the crowd-sourced project because I have a low opportunity cost (in reality I did not participate the project because I was too busy). Nevertheless, that finding highlights the relevance of a "good" journal review process: potential convergence in empirical choices driven by anonymous peer pressure. That also makes me wonder to what extent the "publication pressure" drives empirical choices. For example, if I know that a likely referee absolutely disagrees with the use of a certain empirical method, I might avoid it. So, this might be another source of endogeneity that does not quite show up in the paper's results - dispersion is inflated relative to reality.

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   Good point. The paper also shows that dispersion is similar if the sample is restricted to high-quality scholars

   1 year ago # QUOTE 3 Volod 0 Vlad !

2. Economist
   d849
   

   Harsh take on the paper by AG yesterday.

   details?

   He basically said that dispersion in results is expected given the ambiguity of the hypotheses. For instance, different researchers may have different definitions of market efficiency in mind. But that only strengthens the message of the paper. Weird take.

   You should have in mind that AG’s work is rather sloppy….

   1 year ago # QUOTE 1 Volod 0 Vlad !

3. Economist
   6e0f
   

   Harsh take on the paper by AG yesterday.

   details?

   He basically said that dispersion in results is expected given the ambiguity of the hypotheses. For instance, different researchers may have different definitions of market efficiency in mind. But that only strengthens the message of the paper. Weird take.

   You should have in mind that AG’s work is rather sloppy….

   Any example?

   1 year ago # QUOTE 0 Volod 0 Vlad !

4. Economist
   1b3f
   

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   If you had read the paper:
   Fraction with top econ/finance publications (see footnote 4) 0.31 (research teams) 0.85 (peer evaluator)
   Experience empirical-finance research (low-high, 1-10) 8.1 (1.7) 8.4 (1.8)
   Experience market-liquidity research (low-high, 1-10) 6.9 (2.4) 7.8 (2.3)

   1 year ago # QUOTE 3 Volod 0 Vlad !

5. Economist
   367d
   

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   They controlled for exactly this. Still large variation amongst "high quality" researchers.

   1 year ago # QUOTE 2 Volod 0 Vlad !

6. Economist
   5bb7
   

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   They controlled for exactly this. Still large variation amongst "high quality" researchers.

   This should be pretty obvious anyway from anyone who has ever interacted with so-called "high quality" researchers. Most advanced undergrads have a much better understanding of empirical methods, data, identification than the average 50+ finance prof.

   1 year ago # QUOTE 4 Volod 5 Vlad !

7. Economist
   08ab
   

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   They controlled for exactly this. Still large variation amongst "high quality" researchers.

   This should be pretty obvious anyway from anyone who has ever interacted with so-called "high quality" researchers. Most advanced undergrads have a much better understanding of empirical methods, data, identification than the average 50+ finance prof.

   No identification was needed for this project. Also, have you checked the list of authors?

   1 year ago # QUOTE 1 Volod 1 Vlad !

8. Economist
   49ac
   

   The list of productive young and middle-aged microstructure researchers who did not participate is probably short. I would say it is a pretty representative sample.

   And for those that did not read the paper, there was still huge variation in the results in their subsample of "star" researchers.

   1 year ago # QUOTE 3 Volod 0 Vlad !

9. Economist
   f9a8
   

   Some of the co-authors obviously lack experience according to their ssrn pages. I wonder if co-authors’ findings deviate from “average” by more if they are less experienced. If so, the paper might be exaggerating the issue because lower-ability co-authors may have seen this project as an opportunity to publish; hence, self selection. Put differently, opportunity costs are highere for high-quality researchers. The journal peer review process, on average, can correct for some of this non-standard error as long as the referees are “experts” in the field.

   They controlled for exactly this. Still large variation amongst "high quality" researchers.

   This should be pretty obvious anyway from anyone who has ever interacted with so-called "high quality" researchers. Most advanced undergrads have a much better understanding of empirical methods, data, identification than the average 50+ finance prof.

   Dude, these "high quality" researchers are not your average free-ride deadwoods.... from page 48 " the subset of the highest-quality research teams with highest-quality results (N=9). The teams score one on all of the binary team-quality measures used in Table 3 and at least 7.5 in the experience field (on a 1-10 scale). The reproducibility score of their analysis is at least 75 (out of 100)
   and their average peer evaluator rating is at least 7.5 (out of 10)."

   1 year ago # QUOTE 0 Volod 0 Vlad !

10. Economist
    4d19
    

    4D19 here. On further reflection, I think the main issue with the research design here is that the fincap authors’ incentives differ dramatically from those that drive actual research outcomes. Most importantly, I doubt any serious researcher is participating fincap to score a publication out of this. I am not a believer in rational agents producing public good, let alone when those agents are finance people - this applies to both participants and evaluators. There could be networking incentives, however, for which the actual research output is likely irrelevant. All these would only raise the dispersion on empirical choices and inflate evaluation scores. Overall, there is a likely huge selection bias that would only raise the non-standard errors the paper talks about. We never observe the counter factual non-participants’ outcomes who *chose* not to participate. I would be surprised if a top-5 econ journal wouldn’t desk rejects this. Finance journals might be a different story.

    1 year ago # QUOTE 0 Volod 0 Vlad !

11. Economist
    4d19
    

    ^ that said, I really value the efforts that were put into this. I just doubt we learn much.

    1 year ago # QUOTE 1 Volod 0 Vlad !

12. Economist
    8de7
    

    So everyone gets google scholar cites for this?

    1 year ago # QUOTE 0 Volod 1 Vlad !

13. Economist
    4d19
    

    and ssrn downloads…

    So everyone gets google scholar cites for this?

    1 year ago # QUOTE 0 Volod 1 Vlad !

14. Economist
    f9a8
    

    and ssrn downloads…

    So everyone gets google scholar cites for this?
    

    That's true for every co-authored paper though... it is what it is man

    1 year ago # QUOTE 1 Volod 1 Vlad !

15. Economist
    5bb7
    

    4D19 here. On further reflection, I think the main issue with the research design here is that the fincap authors’ incentives differ dramatically from those that drive actual research outcomes. Most importantly, I doubt any serious researcher is participating fincap to score a publication out of this. I am not a believer in rational agents producing public good, let alone when those agents are finance people - this applies to both participants and evaluators. There could be networking incentives, however, for which the actual research output is likely irrelevant. All these would only raise the dispersion on empirical choices and inflate evaluation scores. Overall, there is a likely huge selection bias that would only raise the non-standard errors the paper talks about. We never observe the counter factual non-participants’ outcomes who *chose* not to participate. I would be surprised if a top-5 econ journal wouldn’t desk rejects this. Finance journals might be a different story.

    You missed the point of the paper completely. There is a large literature by now warning about publication bias and p-hacking. This paper shows that when authors are not incentivized to p-hack (as here is no benefit for finding significant results) there is a wide range of "correct" outcomes. At least "correct" as perceived by experienced finance researchers. This means that empirical researchers who do want to p-hack can choose any of these "correct" results, and tell the story they want to tell. Is this surprising? Not really. Is it important to document and publicize widely among the community? I think so.

    1 year ago # QUOTE 7 Volod 2 Vlad !

16. Economist
    bdd9
    

    We were thought in UG that SEs neglect model uncertainty. We now also have a bunch of Bayesian procedures that explicitly model the model uncertainty. What is the novelty of this paper?

    1 year ago # QUOTE 1 Volod 0 Vlad !

17. Economist
    f2bb
    

    4D19 here. On further reflection, I think the main issue with the research design here is that the fincap authors’ incentives differ dramatically from those that drive actual research outcomes. Most importantly, I doubt any serious researcher is participating fincap to score a publication out of this. I am not a believer in rational agents producing public good, let alone when those agents are finance people - this applies to both participants and evaluators. There could be networking incentives, however, for which the actual research output is likely irrelevant. All these would only raise the dispersion on empirical choices and inflate evaluation scores. Overall, there is a likely huge selection bias that would only raise the non-standard errors the paper talks about. We never observe the counter factual non-participants’ outcomes who *chose* not to participate. I would be surprised if a top-5 econ journal wouldn’t desk rejects this. Finance journals might be a different story.

    You missed the point of the paper completely. There is a large literature by now warning about publication bias and p-hacking. This paper shows that when authors are not incentivized to p-hack (as here is no benefit for finding significant results) there is a wide range of "correct" outcomes. At least "correct" as perceived by experienced finance researchers. This means that empirical researchers who do want to p-hack can choose any of these "correct" results, and tell the story they want to tell. Is this surprising? Not really. Is it important to document and publicize widely among the community? I think so.

    Another point (at least the way I see it) is that even if there's no p-hack at all, there could be a "replication crisis" simply due to high degrees of freedom in empirical research.

    1 year ago # QUOTE 5 Volod 0 Vlad !

18. Economist
    d2a8
    

    4D19 here. On further reflection, I think the main issue with the research design here is that the fincap authors’ incentives differ dramatically from those that drive actual research outcomes. Most importantly, I doubt any serious researcher is participating fincap to score a publication out of this. I am not a believer in rational agents producing public good, let alone when those agents are finance people - this applies to both participants and evaluators. There could be networking incentives, however, for which the actual research output is likely irrelevant. All these would only raise the dispersion on empirical choices and inflate evaluation scores. Overall, there is a likely huge selection bias that would only raise the non-standard errors the paper talks about. We never observe the counter factual non-participants’ outcomes who *chose* not to participate. I would be surprised if a top-5 econ journal wouldn’t desk rejects this. Finance journals might be a different story.

    You missed the point of the paper completely. There is a large literature by now warning about publication bias and p-hacking. This paper shows that when authors are not incentivized to p-hack (as here is no benefit for finding significant results) there is a wide range of "correct" outcomes. At least "correct" as perceived by experienced finance researchers. This means that empirical researchers who do want to p-hack can choose any of these "correct" results, and tell the story they want to tell. Is this surprising? Not really. Is it important to document and publicize widely among the community? I think so.

    This is accurate. What we see as published research is just the outcome one set of choices, selected on the basis of publication likelihood. What the paper shows is precisely how different results could be absent any publication bias.

    1 year ago # QUOTE 5 Volod 0 Vlad !

19. Economist
    d2a8
    

    We were thought in UG that SEs neglect model uncertainty. We now also have a bunch of Bayesian procedures that explicitly model the model uncertainty. What is the novelty of this paper?

    This is more than model uncertainty. There is latitude in deciding what efficiency or liquidity actually mean, in choosing how to measure them, in deciding the relevant sampling frequency, etc.

    1 year ago # QUOTE 2 Volod 0 Vlad !

20. Economist
    bdd9
    

    We were thought in UG that SEs neglect model uncertainty. We now also have a bunch of Bayesian procedures that explicitly model the model uncertainty. What is the novelty of this paper?

    This is more than model uncertainty. There is latitude in deciding what efficiency or liquidity actually mean, in choosing how to measure them, in deciding the relevant sampling frequency, etc.

    Ok, I can understand this. Something like uncertainty of objects that are modeled that precedes the model uncertainty?

    1 year ago # QUOTE 0 Volod 0 Vlad !

Reply »