You didn't get the "appoint a group at the RFS" joke. Since explaining jokes takes the fun away, read this.
Not a bad idea. Ask all authors who had published a paper using Reg SHO to post their data and code.
Ah, yes, you definitely want an insider to look into it. Maybe appoint a group at the RFS to look into it?
Bernie and co can be a little pedantic. Went to their workshop as a grad student. They told me my research design was invalid. Went on to publish it well and win awards on the paper. They are a little too rigid on what they believe is appropriate causal inference and have blind spots as a result.
Bernie and co can be a little pedantic. Went to their workshop as a grad student. They told me my research design was invalid. Went on to publish it well and win awards on the paper. They are a little too rigid on what they believe is appropriate causal inference and have blind spots as a result.
What on earth does "too rigid" mean here? Journals publish all kinds of crap, but you need to stick to certain standards or acknowledge the limitations of your approach.
Bernie and co can be a little pedantic. Went to their workshop as a grad student. They told me my research design was invalid. Went on to publish it well and win awards on the paper. They are a little too rigid on what they believe is appropriate causal inference and have blind spots as a result.What on earth does "too rigid" mean here? Journals publish all kinds of crap, but you need to stick to certain standards or acknowledge the limitations of your approach.
There is a threshold at which you throw out a design due to its limitations rather than continue on acknowledging the limitations as you say and still trying to learn from the model. When I say thy are too rigid, I mean their threshold for throwing out the design is too conservative, and it makes them blind to a lot of good research.
how about the recent AER paper that shows Reg SHO affects CEO compensation contract?
I would recommend BB & co. add this to the list of papers to replicate, just because of where it's placed. I mean the Reg SHO part is a sideshow in the paper (which is mainly theory), but still, it's AER..
The paper is: Contractual Managerial Incentives with Stock Price Feedback by Lin, Liu and Sun
Another fake design like AK’s Russell Index?
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3415529
This is a pre-analysis plan for The Reg SHO Reanalysis Project, in which we will reassess primary results from selected recent accounting and finance papers, which exploit the randomized trial conducted by the Securities and Exchange Commission (SEC) from May 2, 2005 to July 6, 2007. As part of this trial, the SEC selected every third stock on the Russell 3000 index, based on trading volume during the year before the experiment was announced, and exempted these firms’ shares from the short-sale uptick rule and the test. Early studies of the experiment found minor effects on trading markets (e.g., SEC Office of Economic Analysis (2007), Diether et al (2009)), and led the SEC to rescind these short-sale restrictions. However, more recently, a large body of research has documented evidence for a wide variety of outcomes from the experiment, including increased open short interest, negative returns, reduced investment, reduced earnings management, increased audit fees, and much more. We undertook this project because we viewed most of these outcomes as implausible, given the modest expected impact of the short-sale restrictions on trading markets, limited publicity about the experiment (with no opposition from the affected firms that we could find), and early research consistent with small effects. In this proposed project, we will reassess three prominent, recent studies: Fang, Huang and Karpoff (2016), Grullon, Michenaud and Weston (2015), and Hope, Hu, and Zhao (2017). This pre-analysis plan provides details on our sample selection and the specific analyses we plan to carry out for the main results found in these three papers.
Pretty much p-hacking for most “findings”
Another fake design like AK’s Russell Index?
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3415529
This is a pre-analysis plan for The Reg SHO Reanalysis Project, in which we will reassess primary results from selected recent accounting and finance papers, which exploit the randomized trial conducted by the Securities and Exchange Commission (SEC) from May 2, 2005 to July 6, 2007. As part of this trial, the SEC selected every third stock on the Russell 3000 index, based on trading volume during the year before the experiment was announced, and exempted these firms’ shares from the short-sale uptick rule and the test. Early studies of the experiment found minor effects on trading markets (e.g., SEC Office of Economic Analysis (2007), Diether et al (2009)), and led the SEC to rescind these short-sale restrictions. However, more recently, a large body of research has documented evidence for a wide variety of outcomes from the experiment, including increased open short interest, negative returns, reduced investment, reduced earnings management, increased audit fees, and much more. We undertook this project because we viewed most of these outcomes as implausible, given the modest expected impact of the short-sale restrictions on trading markets, limited publicity about the experiment (with no opposition from the affected firms that we could find), and early research consistent with small effects. In this proposed project, we will reassess three prominent, recent studies: Fang, Huang and Karpoff (2016), Grullon, Michenaud and Weston (2015), and Hope, Hu, and Zhao (2017). This pre-analysis plan provides details on our sample selection and the specific analyses we plan to carry out for the main results found in these three papers.
Thanks for the feedback on our Reg SHO project. We plan to, and need to, post our own data and code. We're just not there yet. We hope that a readily accessible, thoroughly cleaned sample, will hep others revisit studies we can't re-examine, just because there are no many of them, with more continuing to come out.
The idea of also looking at Lin et al in AER is an interesting one, and could well make sense for us, despite the extra work involved.
With regard to valid and invalid research designs:
(i) Reg SHO can generate valid designs. Our pre-specified reearch plan lays out one road to credible estimates. Stay tuned for what those estimates will be.
(ii) I personally believe that of the hundreds of IVs in recent finance and accounting studies, maybe a handful are actually valid. For more on that, see Atanasov and Black, The Trouble with Instruments (on SSRN, forthcoming at Management Science), where we study the best of the lot -- those based on external shocks, and find most of them wanting. Necessarily, there must be hundreds of editor/referee pairs that hold IVs to lower standards. No apologies for that!
I'm not a member of the endogeeity police, by the way. It is often a good idea to publish interesting results from an imperfect design, and god knows, I've done that a lot. With disclosure of the design defects, one hopes.
Bernie Black
Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033
Providing the final code does not show that you did not p-hacked (since we dont know the process you got to the final code). But you are in better shape than SK and AB who cant even provide the code.
Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033
Providing the final code does not show that you did not p-hacked (since we dont know the process you got to the final code). But you are in better shape than SK and AB who cant even provide the code.
Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033
They show robustness dips**t
Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033
Glad to see this discussion. Look forward to reading this and any replies from BB and co
I am on Team Black.
Reg monkeys like Huang's and Fans and Zhangs to h*ll.
Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033Glad to see this discussion. Look forward to reading this and any replies from BB and co
From the reply: "We conclude by discussing two theoretical concerns. First, we reiterate that an observed increase in short selling during the Reg SHO period is neither necessary nor sufficient to establish that the prospect of short selling has a disciplinary effect on earnings management, as managers’ endogenous adjustments affect short sellers’ opportunities and observed short selling."
Don't really buy it - I highly doubt most managers were aware that they were affected by reg SHO
Makes little sense. If there is no increase in shorting, what do management worry about?
From the reply: "We conclude by discussing two theoretical concerns. First, we reiterate that an observed increase in short selling during the Reg SHO period is neither necessary nor sufficient to establish that the prospect of short selling has a disciplinary effect on earnings management, as managers’ endogenous adjustments affect short sellers’ opportunities and observed short selling."
Don't really buy it - I highly doubt most managers were aware that they were affected by reg SHO
What have you found? I am curious how many results from reg sho are really there. I have been amazed by the many “effects” on virtually everything for a while.
Thanks for the feedback on our Reg SHO project. We plan to, and need to, post our own data and code. We're just not there yet. We hope that a readily accessible, thoroughly cleaned sample, will hep others revisit studies we can't re-examine, just because there are no many of them, with more continuing to come out.
The idea of also looking at Lin et al in AER is an interesting one, and could well make sense for us, despite the extra work involved.
With regard to valid and invalid research designs:
(i) Reg SHO can generate valid designs. Our pre-specified reearch plan lays out one road to credible estimates. Stay tuned for what those estimates will be.
(ii) I personally believe that of the hundreds of IVs in recent finance and accounting studies, maybe a handful are actually valid. For more on that, see Atanasov and Black, The Trouble with Instruments (on SSRN, forthcoming at Management Science), where we study the best of the lot -- those based on external shocks, and find most of them wanting. Necessarily, there must be hundreds of editor/referee pairs that hold IVs to lower standards. No apologies for that!
I'm not a member of the endogeeity police, by the way. It is often a good idea to publish interesting results from an imperfect design, and god knows, I've done that a lot. With disclosure of the design defects, one hopes.
Bernie Black
You are on Team Black? So which coauthor of Bernie are you?
I am on Team Black.
Reg monkeys like Huang's and Fans and Zhangs to h*ll.Fang, Vivian W. and Huang, Allen and Karpoff, Jonathan M., Reply to 'The Reg SHO Reanalysis Project: Reconsidering Fang, Huang and Karpoff (2016) on Reg SHO and Earnings Management' by Black et al. (2019) (December 19, 2019). Available at SSRN: https://ssrn.com/abstract=3507033Glad to see this discussion. Look forward to reading this and any replies from BB and co