A new marketing / consumer research scandal on sight?

7 months - 2370 posts - Latest - RSS
Thread: 4772 Goods vs 2028 No Goods

Economics Job Market Rumors » Job Market » Industry Rumors

A new marketing / consumer research scandal on sight?

Economist
bedf

Lol. Hi MK.

I think fd28 was saying the effect seems implausible. If it truly shows up in the data without p-hacking then it is likely to be a result of bad experiment design where better subjects were more likely to think experimenter is weird in the weird condition and leave the experiment or disregard rest of the study compared to the ones in the more normal yesterday condition. The different weirdness across conditions could result artificially in this result that has nothing to do with memory inferences or self diagnostocity.

Hmmm. I freakin loved her JMP,
I guess my issue is that you could get significantly different results but you’d misspecify the process. Memory... you should have a model. It’s a constructive process. I’m just not sure this method specifies the process very well, not to be too critical.

Another JCR gem from Kouchaki asking to be replicated:
"You Will Not Remember This: How Memory Efficacy Influences Virtuous Behavior" Maferima Touré-Tillery, Maryam Kouchaki, Journal of Consumer Research, https://doi.org/10.1093/jcr/ucaa023
Participants asked to describe what they were doing exactly 1 month ago rather than yesterday in task 1 donate less money and time at the end of the study because they thought being generous would not be "self-diagnostic"
I also wonder about differential attrition rates in the two conditions. It seems more of make participants do something weird (describe what they were doing exactly 1 month ago; anyone here remember that?) and they will stop paying attention to the study. Infer the experimenter and task is weird, not that they have bad memory (adults are smart enough to know nobody remembers what happened exactly 1 month ago and this is not diagnostic of memory).

Would love to see a replication of this paper!
Ata Jami, Maryam Kouchaki, Francesca Gino, I Own, so I Help Out: How Psychological Ownership Increases Prosocial Behavior, Journal of Consumer Research, , ucaa040, https://doi.org/10.1093/jcr/ucaa040
Past and current OBHDP editors on the same paper? So much POWER!

5 months ago # QUOTE 12 Jab 0 No Jab !
Economist
8f23

All their Princeton/CMU friends are suspicious. none of their research can be replicated. Many of them managed to get tenure in decent places. This is the reality. What can you do?

Wait, this guy (already) has 6 failed replications in his “trophy” cabinet, and without DataColada even getting involved ?!?! CB folks, please tell us this is some kind of record! I’d like to think 6 failed replications (let alone 1 or 2) is the most anyone has accumulated in the CB field. It can’t possibly get worse - Right…?!?!

Surprised no one mentioned AA at NYU, who is good buddies with JS and LN, and so has managed to p-hack his way to job + tenure while the DC crew looked away.
And this despite other folks repeatedly calling his work into question:
http://andrewgelman.com/2014/11/24/oh-go/
https://jasoncollins.org/2016/05/11/bad-behavioural-science-failures-bias-and-fairy-tales/
http://www.terryburnham.com/2015/04/a-trick-for-higher-sat-scores.html
http://www.chabris.com/ChabrisGladwellReviewWSJ.pdf
https://doi.org/10.1017/S0140525X15000965
And multiple failures (by several different research teams) to replicate his work:
1. https://doi.org/10.1016/j.cognition.2012.09.012
2. https://doi.org/10.1037/xge0000049
3. https://doi.org/10.1073/pnas.1424903112
4. https://doi.org/10.1016/j.cognition.2019.104156
5. https://doi.org/10.1073/pnas.1516179112
6. https://www.usenix.org/laser2013/program/balebako
And that’s before we get to these additional ‘gems’, which (to my knowledge) have yet to be examined:
Alter, A. L., Stern. C., Granot, Y., & Balcetis, E. (2016). The “bad is black” effect: Why people believe evildoers have darker skin than do-gooders. Personality and Social Psychology Bulletin, 42, 1653-1665.
Alter, A. (2013). When in Chinatown, You Really Do Think More Chinese. Harvard Business Review, 91(3), 28-29.
Alter, A. L., & Balcetis, E. (2011). Fondness makes the distance grow shorter: Desired locations seem closer because they are more vivid. Journal of Experimental Social Psychology, 47, 16-21.
Alter, A. L., & Kwan, V. S. Y. (2009). Cultural sharing in a global village: Extracultural cognition in European Americans. Journal of Personality and Social Psychology, 96, 742-760.
Alter, A. L., & Oppenheimer, D. M. (2008). Easy on the mind, easy on the wallet: Effects of fluency on valuation judgments. Psychonomic Bulletin and Review, 15, 985-990.
In sum, if AA’s work were the embodiment of a priming study, one can’t help but think it might be this one:
https://doi.org/10.1037/a0029708
(*all puns intended*)
The DC silence may be explained by the fact that AA is part of the Princeton clique, and also that he promotes their work in the popular media:
https://www.newyorker.com/tech/elements/the-power-of-name ...See full post

5 months ago # QUOTE 3 Jab 8 No Jab !
Economist
a5ee

Relentless DC troll: "none of their research can be replicated" (where is your evidence for this?!)

Princeton/CMU clique:

Galak, J., LeBoeuf, R. A., Nelson, L. D., & Simmons, J. P. (2012). Correcting the past: Failures to replicate psi. Journal of Personality and Social Psychology, 103(6), 933–948. https://doi.org/10.1037/a0029709

Seriously, just stop wasting everyone's time by constantly trying to divert attention and defame people who *actually* run replications.

All their Princeton/CMU friends are suspicious. none of their research can be replicated. Many of them managed to get tenure in decent places. This is the reality. What can you do?

Wait, this guy (already) has 6 failed replications in his “trophy” cabinet, and without DataColada even getting involved ?!?! CB folks, please tell us this is some kind of record! I’d like to think 6 failed replications (let alone 1 or 2) is the most anyone has accumulated in the CB field. It can’t possibly get worse - Right…?!?!

Surprised no one mentioned AA at NYU, who is good buddies with JS and LN, and so has managed to p-hack his way to job + tenure while the DC crew looked away.
And this despite other folks repeatedly calling his work into question:
http://andrewgelman.com/2014/11/24/oh-go/
https://jasoncollins.org/2016/05/11/bad-behavioural-science-failures-bias-and-fairy-tales/
http://www.terryburnham.com/2015/04/a-trick-for-higher-sat-scores.html
http://www.chabris.com/ChabrisGladwellReviewWSJ.pdf
https://doi.org/10.1017/S0140525X15000965
And multiple failures (by several different research teams) to replicate his work:
1. https://doi.org/10.1016/j.cognition.2012.09.012
2. https://doi.org/10.1037/xge0000049
3. https://doi.org/10.1073/pnas.1424903112
4. https://doi.org/10.1016/j.cognition.2019.104156
5. https://doi.org/10.1073/pnas.1516179112
6. https://www.usenix.org/laser2013/program/balebako
And that’s before we get to these additional ‘gems’, which (to my knowledge) have yet to be examined:
Alter, A. L., Stern. C., Granot, Y., & Balcetis, E. (2016). The “bad is black” effect: Why people believe evildoers have darker skin than do-gooders. Personality and Social Psychology Bulletin, 42, 1653-1665.
Alter, A. (2013). When in Chinatown, You Really Do Think More Chinese. Harvard Business Review, 91(3), 28-29.
Alter, A. L., & Balcetis, E. (2011). Fondness makes the distance grow shorter: Desired locations seem closer because they are more vivid. Journal of Experimental Social Psychology, 47, 16-21.
Alter, A. L., & Kwan, V. S. Y. (2009). Cultural sharing in a global village: Extracultural cognition in European Americans. Journal of Personality and Social Psychology, 96, 742-760.
Alter, A. L., & Oppenheimer, D. M. (2008). Easy on the mind, easy on the wallet: Effects of fluency on valuation judgments. Psychonomic...See full post

5 months ago # QUOTE 7 Jab 3 No Jab !
Economist
1a6f

IH & TH's research look suspicious too ... anyone requested data or replicated their work?

And XH's coathor TH at HKU seems to be too productive.
https://www.fbe.hku.hk/people/academic/tak-zhongqiang-huang
XH, TH, and PD all graduated from CUHK.
Any secrets there?

Hmm. Let's see. According to this CV (http://marketingphdjobs.org/sites/default/files/Huang,%20Zhongqiang%20(Tak)_CV%20Short%20Sep16.pdf), he got his Ph.D. in 2014, and was a Postdoctoral researcher until he joined HKU in 2017. So it's been 6 years since he got his Ph.D. and he has six publications. 2 JMRs, 2 JCRs, 1 IJRM, and 1 JESP.
Now, I will show you another person who I think it is extremely productive. I'm not suggesting anything, but we need to have some standards of comparisons to set what is a productive person. Let's see SM from Wharton.
https://marketing.wharton.upenn.edu/profile/_whrtn71/#
She's got her PhD in 2017. It's been 3 years since she's got her Ph.D.
https://scholar.google.com/citations?hl=en&user=1UnvjWoAAAAJ&view_op=list_works&sortby=pubdate
1 JM (!), 2 JMRs, 2 JCRs, 1 JCP, 1 CNS...
It's quite a lot. So she's "twice" as productive as TH, I guess? It's difficult to categorize what it means to be "too productive".

5 months ago # QUOTE 1 Jab 10 No Jab !
Economist
f7bf

I meant MTT’s JMP. It was awesome.

Anyway I basically agree with you. It’s not clear they’re measuring what they claim to measure. Less of a p hacking problem than a specification one.

Lol. Hi MK.
I think fd28 was saying the effect seems implausible. If it truly shows up in the data without p-hacking then it is likely to be a result of bad experiment design where better subjects were more likely to think experimenter is weird in the weird condition and leave the experiment or disregard rest of the study compared to the ones in the more normal yesterday condition. The different weirdness across conditions could result artificially in this result that has nothing to do with memory inferences or self diagnostocity.

Hmmm. I freakin loved her JMP,
I guess my issue is that you could get significantly different results but you’d misspecify the process. Memory... you should have a model. It’s a constructive process. I’m just not sure this method specifies the process very well, not to be too critical.

Another JCR gem from Kouchaki asking to be replicated:
"You Will Not Remember This: How Memory Efficacy Influences Virtuous Behavior" Maferima Touré-Tillery, Maryam Kouchaki, Journal of Consumer Research, https://doi.org/10.1093/jcr/ucaa023
Participants asked to describe what they were doing exactly 1 month ago rather than yesterday in task 1 donate less money and time at the end of the study because they thought being generous would not be "self-diagnostic"
I also wonder about differential attrition rates in the two conditions. It seems more of make participants do something weird (describe what they were doing exactly 1 month ago; anyone here remember that?) and they will stop paying attention to the study. Infer the experimenter and task is weird, not that they have bad memory (adults are smart enough to know nobody remembers what happened exactly 1 month ago and this is not diagnostic of memory).

Would love to see a replication of this paper!
Ata Jami, Maryam Kouchaki, Francesca Gino, I Own, so I Help Out: How Psychological Ownership Increases Prosocial Behavior, Journal of Consumer Research, , ucaa040, https://doi.org/10.1093/jcr/ucaa040
Past and current OBHDP editors on the same paper? So much POWER!

5 months ago # QUOTE 0 Jab 12 No Jab !
Economist
8304

He just got promoted to Full Professor (per a post on Twitter).

Wait, this guy (already) has 6 failed replications in his “trophy” cabinet, and without DataColada even getting involved ?!?! CB folks, please tell us this is some kind of record! I’d like to think 6 failed replications (let alone 1 or 2) is the most anyone has accumulated in the CB field. It can’t possibly get worse - Right…?!?!

Surprised no one mentioned AA at NYU, who is good buddies with JS and LN, and so has managed to p-hack his way to job + tenure while the DC crew looked away.
And this despite other folks repeatedly calling his work into question:
http://andrewgelman.com/2014/11/24/oh-go/
https://jasoncollins.org/2016/05/11/bad-behavioural-science-failures-bias-and-fairy-tales/
http://www.terryburnham.com/2015/04/a-trick-for-higher-sat-scores.html
http://www.chabris.com/ChabrisGladwellReviewWSJ.pdf
https://doi.org/10.1017/S0140525X15000965
And multiple failures (by several different research teams) to replicate his work:
1. https://doi.org/10.1016/j.cognition.2012.09.012
2. https://doi.org/10.1037/xge0000049
3. https://doi.org/10.1073/pnas.1424903112
4. https://doi.org/10.1016/j.cognition.2019.104156
5. https://doi.org/10.1073/pnas.1516179112
6. https://www.usenix.org/laser2013/program/balebako
And that’s before we get to these additional ‘gems’, which (to my knowledge) have yet to be examined:
Alter, A. L., Stern. C., Granot, Y., & Balcetis, E. (2016). The “bad is black” effect: Why people believe evildoers have darker skin than do-gooders. Personality and Social Psychology Bulletin, 42, 1653-1665.
Alter, A. (2013). When in Chinatown, You Really Do Think More Chinese. Harvard Business Review, 91(3), 28-29.
Alter, A. L., & Balcetis, E. (2011). Fondness makes the distance grow shorter: Desired locations seem closer because they are more vivid. Journal of Experimental Social Psychology, 47, 16-21.
Alter, A. L., & Kwan, V. S. Y. (2009). Cultural sharing in a global village: Extracultural cognition in European Americans. Journal of Personality and Social Psychology, 96, 742-760.
Alter, A. L., & Oppenheimer, D. M. (2008). Easy on the mind, easy on the wallet: Effects of fluency on valuation judgments. Psychonomic Bulletin and Review, 15, 985-990.
In sum, if AA’s work were the embodiment of a priming study, one can’t help but think it might be this one:
https://doi.org/10.1037/a0029708
(*all puns intended*)
The DC silence may be explained by the fact that AA is part of the Princeton clique, and also that he promotes their work in the popular media:
https://www.newyorker.com/tech/elements/the-power-of-names
CB research failing to replicate is no surprise - lots of it is too ridiculous to be true - but 6 differen...See full post

5 months ago # QUOTE 0 Jab 1 No Jab !
Economist
c705

All my research uses Bayes, and has for a long time. You are referring to the "evidence". Which serious researchers have suggested using this for inference about null hypotheses? It should be easy to post a link or two. I agree that Bayes can reflect the relative evidence of any two models, but not that it would lead one to actually accept the null.

Anyone who's serious about testing the relative strength of models should be using some form of cross-validation, like k-fold or leave-on-out. This is completely mechanical, if time-consuming, and can be done by using the entire posterior from each left-out sample.

The original remark about using Bayes to accept nulls just gives people the wrong idea.

I agree with the last post here, but I have an addition.
Bayesian actually can test for nulls for the following reason: Bayesian creates a null model, and then tests how the addition of your 'manipulation' or conditions in the design impacts the predictable power of the model. If the conditions are above 3 times more likely than the null, then there is modest evidence that your manipulation is indeed doing something. On the other hand, if the null model is about 3 times more likely than your conditions to explain variations in your DV, then it is likely that your conditions are showing a null effect. THe only time the evidence is inconclusive is when your model is around 1 to 1 with the null model in explaining the variations in the DV.
If you read some recent articles on Bayes, you would find all these implications and more (including the fact that Bayes allows for smaller samples as it can test for different assumptions, different effect sizes, and can do sequential analysis which allows you to predict via simulation whether your effect would get a better predictive power with more participants). Pretty cool stuff.

WHAT? Don't say foolish stuff. Bayes doesn't "demonstrate" nulls. All it does in practice is allow you to dispense with various assumptions baked into frequentist analyses, especially in small samples. And to build your own models instead of relying on SAS or whatever.

5 months ago # QUOTE 1 Jab 0 No Jab !
Economist
d555

NN, too? He seems stalled.

All their Princeton/CMU friends are suspicious. none of their research can be replicated. Many of them managed to get tenure in decent places. This is the reality. What can you do?

Wait, this guy (already) has 6 failed replications in his “trophy” cabinet, and without DataColada even getting involved ?!?! CB folks, please tell us this is some kind of record! I’d like to think 6 failed replications (let alone 1 or 2) is the most anyone has accumulated in the CB field. It can’t possibly get worse - Right…?!?!

Surprised no one mentioned AA at NYU, who is good buddies with JS and LN, and so has managed to p-hack his way to job + tenure while the DC crew looked away.
And this despite other folks repeatedly calling his work into question:
http://andrewgelman.com/2014/11/24/oh-go/
https://jasoncollins.org/2016/05/11/bad-behavioural-science-failures-bias-and-fairy-tales/
http://www.terryburnham.com/2015/04/a-trick-for-higher-sat-scores.html
http://www.chabris.com/ChabrisGladwellReviewWSJ.pdf
https://doi.org/10.1017/S0140525X15000965
And multiple failures (by several different research teams) to replicate his work:
1. https://doi.org/10.1016/j.cognition.2012.09.012
2. https://doi.org/10.1037/xge0000049
3. https://doi.org/10.1073/pnas.1424903112
4. https://doi.org/10.1016/j.cognition.2019.104156
5. https://doi.org/10.1073/pnas.1516179112
6. https://www.usenix.org/laser2013/program/balebako
And that’s before we get to these additional ‘gems’, which (to my knowledge) have yet to be examined:
Alter, A. L., Stern. C., Granot, Y., & Balcetis, E. (2016). The “bad is black” effect: Why people believe evildoers have darker skin than do-gooders. Personality and Social Psychology Bulletin, 42, 1653-1665.
Alter, A. (2013). When in Chinatown, You Really Do Think More Chinese. Harvard Business Review, 91(3), 28-29.
Alter, A. L., & Balcetis, E. (2011). Fondness makes the distance grow shorter: Desired locations seem closer because they are more vivid. Journal of Experimental Social Psychology, 47, 16-21.
Alter, A. L., & Kwan, V. S. Y. (2009). Cultural sharing in a global village: Extracultural cognition in European Americans. Journal of Personality and Social Psychology, 96, 742-760.
Alter, A. L., & Oppenheimer, D. M. (2008). Easy on the mind, easy on the wallet: Effects of fluency on valuation judgments. Psychonomic Bulletin and Review, 15, 985-990.
In sum, if AA’s work were the embodiment of a priming study, one can’t help but think it might be this one:
https://doi.org/10.1037/a0029708
(*all puns intended*)
The DC silence may be explained by the fact that AA is part of the Princeton clique, and also that he promotes their work in the popular media:
https:...See full post

5 months ago # QUOTE 1 Jab 2 No Jab !
Economist
f7bf

I meant MTT’s JMP. It was awesome.
Anyway I basically agree with you. It’s not clear they’re measuring what they claim to measure. Less of a p hacking problem than a specification one.

Lol. Hi MK.
I think fd28 was saying the effect seems implausible. If it truly shows up in the data without p-hacking then it is likely to be a result of bad experiment design where better subjects were more likely to think experimenter is weird in the weird condition and leave the experiment or disregard rest of the study compared to the ones in the more normal yesterday condition. The different weirdness across conditions could result artificially in this result that has nothing to do with memory inferences or self diagnostocity.

Hmmm. I freakin loved her JMP,
I guess my issue is that you could get significantly different results but you’d misspecify the process. Memory... you should have a model. It’s a constructive process. I’m just not sure this method specifies the process very well, not to be too critical.

Lotta dislike for MTT wow

5 months ago # QUOTE 0 Jab 8 No Jab !
Economist
d9ed

Read the entire post. It is about 2 recent implausible JCRs by AJ, MK, FG and by MTT, MK. This is after DC could not replicate a third trashy JCR by AJ. So the focus was on trashy recent JCRs that have not replicated or are unlikely to replicate because of points that are fair or on a network if people starting with AJ and the original DC failed paper replication. Until someone decided to deflect by randomly praising MTTs dissertation.

I meant MTT’s JMP. It was awesome.
Anyway I basically agree with you. It’s not clear they’re measuring what they claim to measure. Less of a p hacking problem than a specification one.

Lol. Hi MK.
I think fd28 was saying the effect seems implausible. If it truly shows up in the data without p-hacking then it is likely to be a result of bad experiment design where better subjects were more likely to think experimenter is weird in the weird condition and leave the experiment or disregard rest of the study compared to the ones in the more normal yesterday condition. The different weirdness across conditions could result artificially in this result that has nothing to do with memory inferences or self diagnostocity.

Hmmm. I freakin loved her JMP,
I guess my issue is that you could get significantly different results but you’d misspecify the process. Memory... you should have a model. It’s a constructive process. I’m just not sure this method specifies the process very well, not to be too critical.

Lotta dislike for MTT wow

5 months ago # QUOTE 10 Jab 0 No Jab !
Economist
093b

Here is a few articles for 'testing the null' with Bayes:

Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic bulletin & review, 16(2), 225-237.

Hoekstra, R., Monden, R., van Ravenzwaaij, D., & Wagenmakers, E. J. (2018). Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects. PloS one, 13(4), e0195474.

Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in psychology, 5, 781.

Leppink, J., O’sullivan, P., & Winston, K. (2017). Evidence against vs. in favour of a null hypothesis. Perspectives on medical education, 6(2), 115-118.

All my research uses Bayes, and has for a long time. You are referring to the "evidence". Which serious researchers have suggested using this for inference about null hypotheses? It should be easy to post a link or two. I agree that Bayes can reflect the relative evidence of any two models, but not that it would lead one to actually accept the null.
Anyone who's serious about testing the relative strength of models should be using some form of cross-validation, like k-fold or leave-on-out. This is completely mechanical, if time-consuming, and can be done by using the entire posterior from each left-out sample.
The original remark about using Bayes to accept nulls just gives people the wrong idea.

I agree with the last post here, but I have an addition.
Bayesian actually can test for nulls for the following reason: Bayesian creates a null model, and then tests how the addition of your 'manipulation' or conditions in the design impacts the predictable power of the model. If the conditions are above 3 times more likely than the null, then there is modest evidence that your manipulation is indeed doing something. On the other hand, if the null model is about 3 times more likely than your conditions to explain variations in your DV, then it is likely that your conditions are showing a null effect. THe only time the evidence is inconclusive is when your model is around 1 to 1 with the null model in explaining the variations in the DV.
If you read some recent articles on Bayes, you would find all these implications and more (including the fact that Bayes allows for smaller samples as it can test for different assumptions, different effect sizes, and can do sequential analysis which allows you to predict via simulation whether your effect would get a better predictive power with more participants). Pretty cool stuff.

WHAT? Don't say foolish stuff. Bayes doesn't "demonstrate" nulls. All it does in practice is allow you to dispense with various assumptions baked into frequentist analyses, especially in small samples. And to build your own models instead of relying on SAS or whatever.

5 months ago # QUOTE 3 Jab 1 No Jab !
Economist
f7bf

I’m just saying it’s less p hacking than they probably got an implausible result and then tortured some theory post hoc and put it at the front of the paper.

Is that p hacking? Is that t hacking (McGraw and Williams)? I dunno. Seems bad but a different issue.
It pains me to see someone I respect a ton engage in the behavior. It’s one of the worst things people do in the field because they’re pressured to put a theory on an effect even when there’s no way it’s plausible. Otherwise, it won’t get into JCR.

Read the entire post. It is about 2 recent implausible JCRs by AJ, MK, FG and by MTT, MK. This is after DC could not replicate a third trashy JCR by AJ. So the focus was on trashy recent JCRs that have not replicated or are unlikely to replicate because of points that are fair or on a network if people starting with AJ and the original DC failed paper replication. Until someone decided to deflect by randomly praising MTTs dissertation.

I meant MTT’s JMP. It was awesome.
Anyway I basically agree with you. It’s not clear they’re measuring what they claim to measure. Less of a p hacking problem than a specification one.

5 months ago # QUOTE 0 Jab 1 No Jab !
Economist
2a14

I believe you were talking about:

https://journals.sagepub.com/doi/abs/10.1207/s15327957pspr0203_4

I’m just saying it’s less p hacking than they probably got an implausible result and then tortured some theory post hoc and put it at the front of the paper.
Is that p hacking? Is that t hacking (McGraw and Williams)? I dunno. Seems bad but a different issue.
It pains me to see someone I respect a ton engage in the behavior. It’s one of the worst things people do in the field because they’re pressured to put a theory on an effect even when there’s no way it’s plausible. Otherwise, it won’t get into JCR.

5 months ago # QUOTE 0 Jab 1 No Jab !
Economist
2a14

Both p-hacking and HARKing are questionable research practices that inflate type I error rate by abusing researcher degrees of freedom.

Incidentally, direct replication is an antidote to both. Note that with “conceptual replications” one can always add post hoc ancillary hypotheses (“hidden moderators”) to rationalize a failed replication. Which is why direct replications are all the more important.

I believe you were talking about:
https://journals.sagepub.com/doi/abs/10.1207/s15327957pspr0203_4

I’m just saying it’s less p hacking than they probably got an implausible result and then tortured some theory post hoc and put it at the front of the paper.
Is that p hacking? Is that t hacking (McGraw and Williams)? I dunno. Seems bad but a different issue.
It pains me to see someone I respect a ton engage in the behavior. It’s one of the worst things people do in the field because they’re pressured to put a theory on an effect even when there’s no way it’s plausible. Otherwise, it won’t get into JCR.

5 months ago # QUOTE 4 Jab 0 No Jab !
Economist
01fc

f7bf is confirmed AJ. His first post deflects attention from his +1 MK to MTT by randomly praising the JMP. When the post returns to the JCR he again deflects to the JMP. Then he puts a third post about dislike for MTT to deflect from the three JCR being discussed. Then he puts in this bullish!t. Hahaha

Both p-hacking and HARKing are questionable research practices that inflate type I error rate by abusing researcher degrees of freedom.
Incidentally, direct replication is an antidote to both. Note that with “conceptual replications” one can always add post hoc ancillary hypotheses (“hidden moderators”) to rationalize a failed replication. Which is why direct replications are all the more important.

I believe you were talking about:
https://journals.sagepub.com/doi/abs/10.1207/s15327957pspr0203_4

I’m just saying it’s less p hacking than they probably got an implausible result and then tortured some theory post hoc and put it at the front of the paper.
Is that p hacking? Is that t hacking (McGraw and Williams)? I dunno. Seems bad but a different issue.
It pains me to see someone I respect a ton engage in the behavior. It’s one of the worst things people do in the field because they’re pressured to put a theory on an effect even when there’s no way it’s plausible. Otherwise, it won’t get into JCR.

5 months ago # QUOTE 1 Jab 0 No Jab !
Economist
f7bf

Who is AJ? I think I’ve dropped enough crumbs so you could figure out who I am. (I have no opinion on MK’s work other than general agreement on the surface sketchiness, just like MTT’s work, a lot, generally)

f7bf is confirmed AJ. His first post deflects attention from his +1 MK to MTT by randomly praising the JMP. When the post returns to the JCR he again deflects to the JMP. Then he puts a third post about dislike for MTT to deflect from the three JCR being discussed. Then he puts in this bullish!t. Hahaha

Both p-hacking and HARKing are questionable research practices that inflate type I error rate by abusing researcher degrees of freedom.
Incidentally, direct replication is an antidote to both. Note that with “conceptual replications” one can always add post hoc ancillary hypotheses (“hidden moderators”) to rationalize a failed replication. Which is why direct replications are all the more important.

I believe you were talking about:
https://journals.sagepub.com/doi/abs/10.1207/s15327957pspr0203_4

5 months ago # QUOTE 0 Jab 0 No Jab !
Economist
f7bf

Direct replications should follow a conceptual replication. Conceptual replications represent an advancement in specifying the construct under measurement. One without the other doesn’t really help us understand the construct, and in the case of failed replication makes it easy to dodge (Hidden moderators, eg).

There’s also the JL critique of direct replications, but I know a bit less about that one.

Both p-hacking and HARKing are questionable research practices that inflate type I error rate by abusing researcher degrees of freedom.
Incidentally, direct replication is an antidote to both. Note that with “conceptual replications” one can always add post hoc ancillary hypotheses (“hidden moderators”) to rationalize a failed replication. Which is why direct replications are all the more important.

I believe you were talking about:
https://journals.sagepub.com/doi/abs/10.1207/s15327957pspr0203_4

I’m just saying it’s less p hacking than they probably got an implausible result and then tortured some theory post hoc and put it at the front of the paper.
Is that p hacking? Is that t hacking (McGraw and Williams)? I dunno. Seems bad but a different issue.
It pains me to see someone I respect a ton engage in the behavior. It’s one of the worst things people do in the field because they’re pressured to put a theory on an effect even when there’s no way it’s plausible. Otherwise, it won’t get into JCR.

5 months ago # QUOTE 0 Jab 2 No Jab !
Economist
f7bf

Oh lord AJ wrote the elevated viewpoints paper. Woooooollolfff that’s bad

5 months ago # QUOTE 2 Jab 0 No Jab !
Economist
c705

Thanks for following through. I will check these out. They are all outside stats journals, though, so I'm guessing they are focused on applications rather than theory.

Here is a few articles for 'testing the null' with Bayes:
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic bulletin & review, 16(2), 225-237.
Hoekstra, R., Monden, R., van Ravenzwaaij, D., & Wagenmakers, E. J. (2018). Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects. PloS one, 13(4), e0195474.
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in psychology, 5, 781.
Leppink, J., O’sullivan, P., & Winston, K. (2017). Evidence against vs. in favour of a null hypothesis. Perspectives on medical education, 6(2), 115-118.

5 months ago # QUOTE 1 Jab 0 No Jab !
Economist
f7bf

PB&R is legit

Thanks for following through. I will check these out. They are all outside stats journals, though, so I'm guessing they are focused on applications rather than theory.

Here is a few articles for 'testing the null' with Bayes:
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic bulletin & review, 16(2), 225-237.
Hoekstra, R., Monden, R., van Ravenzwaaij, D., & Wagenmakers, E. J. (2018). Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects. PloS one, 13(4), e0195474.
Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in psychology, 5, 781.
Leppink, J., O’sullivan, P., & Winston, K. (2017). Evidence against vs. in favour of a null hypothesis. Perspectives on medical education, 6(2), 115-118.

5 months ago # QUOTE 0 Jab 0 No Jab !

« Previous 1…585960…119 Next »

Recently Announced

Upcoming

Economics

Job Market

Off Topic

Job Market Wiki

Job Market Candidates

Updates

A new marketing / consumer research scandal on sight?

Reply »