Oster: absence of evidence is evidence of absence
Emily Oster, comment?
-
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book.Evidence that something causes harm to the fetus is questionable. That must mean it is safe and I should do it!
-
This person gets it! Fellow parent econ PhD ..
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book. -
What Dumbo doctors apologists post here?
Med research is entirely phacking on observational studies.
Doctors in hospitals just follow rules.Doctors consider various levels of evidence, and are completely aware of the weaknesses of observational studies. They actually conduct randomized experiments, which they consider to be amongst the highest evidence for something. Doctors all know that observational studies are provide weaker evidence.
In fact, you can argue that doctors are even more stringent than economists in how they evaluate studies. The belief amongst doctors is that causality cannot be established outside of randomized trials. Compare that to economists who bloviate about the identification revolution and use appeals to authority or cute arguments to claim causality for their identification approach.This is simply not true. Do you actually talk to doctors?
In all of my medical interactions, they did NOT weigh different evidence. They cited "hospital policy" on every single issue,with no regard to any evidence.
-
The fact the general public thinks like this shows the total failure of scientific education in America.The author says it's fine to drink some alcohol during pregnancy, and details how much and why she thinks this. She walks through studies and literature on the topic that she examined. Given that the CDC, the American Pregnancy Association, the World Health Organization, National Council on Alcoholism and Drug Dependence, the March of Dimes, and others all agree and firmly say the best advice is to abstain from alcohol completely, I really needed an extraordinary explanation from the author as to why I should ignore those experts. Not surprisingly, I found her analysis underwhelming and unpersuasive, and not worthy of ignoring the consensus shared by experts in the field.
Ask yourself, who is a more trusted source? An economist writing a book to a mass audience for profit, or the March of Dimes, the CDC, WHO, American Pregnancy Association, the UK Health Ministry, Fetal Alcohol Syndrome scientists and advocacy groups? Do all those independent groups have a profit motive to give distorted and wrong health advice?
."The studies say one thing, but public health groups say another - therefore we should trust the public health groups".
What? The studies are literally all the relevant information available. Any view that isnt based on them is inrinsically bulls**t. Its not like doctors and public health groups/WHO/etc have access to secret information that isnt being made public. If their advice contradicts the actual scientific literature then their advice is wrong, period.
How on earth did this trainwreck of a post get so many upvotes on a forum of alleged PhDs?
-
What you ignore (and will continue to ignore, I know) is the experience of your doctor gained over the years seeing hundreds of patients. That is something a good doctor relies on much more while making recommendations, than reading studies or medical literature. One reason for this maybe is that you do not have a good doctor yourself (the other person saying doctors quote hospital rules is in the same category). I would suggest you to look around nd get a good doctor, one you can trust. I have one - I go to a micro-practice and have one who I highly respect for her level of intuition and knowledge - and trust completely.
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book. -
This is 100% correct.Have any of you ever interacted with doctors? They no zero stats, no none of the relevant literature, and make absurd recommendations dressed up as holy truths. I'd trust the average second year PhD student with two hours on Google scholar more on medical decisions than the team at my University's elite teaching hospital
In fairness, there is no big reason for physicians to know a lot of stats - they arent researchers, and they should be able to just trust the relevant medical/health literature. So I wouldnt blame doctors/physicians at all.
The real problem is that public health is a trainwreck of a field where most researchers have almost no stats training beyond OLS. Almost none of these people have any real idea about how to study causality, and 90% of papers have the form "X is correlated with Y, this correlation still holds after we control for a few observable demographic variables, therefore we can conclude that X causes Y".
The field is just a total mess and almost none of these idiots has any idea what they are doing. I dont think economists realise quite how bad it is, its basically as if you had vital medical research carried out by sociologists.
-
Ask yourself, who is a more trusted source? An economist writing a book to a mass audience for profit, or the March of Dimes, the CDC, WHO, American Pregnancy Association, the UK Health Ministry, Fetal Alcohol Syndrome scientists and advocacy groups?
Is this a joke? I'll take the modal economist of Oster's vintage over all of those groups any day. Most doctors, medical researchers, and "advocacy groups" do not have a clue about identification or causal inference. As you would know if you'd ever read BMJ or JAMA.
Oster may be wrong in this case. And I am risk–averse and wouldn't drink while pregnant. But that hardly means that Oster is a worse source of information in general.
Doctors consider various levels of evidence, and are completely aware of the weaknesses of observational studies. They actually conduct randomized experiments, which they consider to be amongst the highest evidence for something. Doctors all know that observational studies are provide weaker evidence.
But a lot of folk wisdom about pregnancy and much else comes from observational studies. And even when doctors run experiments, they are badly underpowered and often mis-analyzed. They also pay about zero attention to the magnitude of the effects that they're investigating. (Are they small? Large? Who cares, as long as the estimate is significant?)
The lack of power might not be such a grave problem if the results weren't translated into unequivocal lessons, if the medical community communicated its results with the appropriate level of uncertainty. But they don't.
-
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book.Evidence that something causes harm to the fetus is questionable. That must mean it is safe and I should do it!
You are an idiot, but I am going to answer you anyway.
Ironically, what you are sarcastically accusing me of is exactly what some doctor's recommendations do. Which OTC drugs are safe for the mother to take during pregnancy? As Oster describes in the book, the risk categories ask whether there is evidence of harm to the fetus. Of course, a lack of evidence could either be because any adverse effect is small/nonexistant, or because it is actually is harmful but the studies in the literature are not high-powered enough to detect it (or suffer from other biases that attenuate the evidence). The risk categorization does not distinguish these. Literally, it is based on p<.05.
-
You've really drunk the kool-aid, def7. I don't know who you are, but if I did I would definitely check and recheck your appendices as a referee to be sure you didn't rely on "experience" and "intuition" to get your results.
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book.Evidence that something causes harm to the fetus is questionable. That must mean it is safe and I should do it!
What you ignore (and will continue to ignore, I know) is the experience of your doctor gained over the years seeing hundreds of patients. That is something a good doctor relies on much more while making recommendations, than reading studies or medical literature. One reason for this maybe is that you do not have a good doctor yourself (the other person saying doctors quote hospital rules is in the same category). I would suggest you to look around nd get a good doctor, one you can trust. I have one - I go to a micro-practice and have one who I highly respect for her level of intuition and knowledge - and trust completely.
Personally, being a parent drove me several times to read medical literature on one issue or another we were facing during pregnancy. I found the quality of these studies to be shockingly low. The empirics are flawed (small sample sizes, selected samples, omitted variables, serious survey biases like recall), there are no robustness checks, the summary statistics are of limited usefulness or aren't presented at all, and on and on. (I found myself really appreciating the length of econ papers.) I'd be happy to give a couple of specific examples if you are interested. I suspect most here are just trolls looking to get some punches in on their favorite EJMR villain.
Recommendations need to be made to parents, so the flimsy studies must be relied upon and extrapolated from. Remember that doctors do not necessarily read the studies, but rather follow the cookbook they find in their medical texts. Also, part of the issue is that most parents couldn't handle being presented any nuance or handle any uncertainty behind the recommendations.
In the end, I don't really believe any of the recommendations, and I now feel like I need to read the literature myself. Of course this is suboptimal, as I am not a medical expert. I appreciated that Emily did some of the legwork for me in the book. -
What you ignore (and will continue to ignore, I know) is the experience of your doctor gained over the years seeing hundreds of patients. That is something a good doctor relies on much more while making recommendations, than reading studies or medical literature. One reason for this maybe is that you do not have a good doctor yourself (the other person saying doctors quote hospital rules is in the same category). I would suggest you to look around nd get a good doctor, one you can trust. I have one - I go to a micro-practice and have one who I highly respect for her level of intuition and knowledge - and trust completely.
The problem with this assertion is that there is a lack of feedback in the process. Adverse events are infrequent enough, and have enough causes, that I don't think a doctor can translate past experience into intuition.
-
I do not want to get personal here, but there is something I will say as a matter of full disclosure (since that may be driving my comments a wee bit). As a rational decision-maker, I take into account all available information, which includes information about Emily's past ethics as a researcher and the research ethics of her mentors and reviewer network. At a minimum, there is a certain carelessness involved. Given this information, I discount her findings, especially with respect to findings that will impact my unborn child. This is completely consistent with rational behavior. What drives the rationality of posters who do not take into account the information about EOs research ethics, but question the research practices of the medical community? And use this sloppy rationalization to make important life decisions. I would have no hesitation using recommendations formed on the basis of the work another researcher whose research ethics I trust (like Anna in the same department). And I am a theorist who does not have time or training to carefully carry out a complete analysis in this specific field myself.
-
What Dumbo doctors apologists post here?
Med research is entirely phacking on observational studies.
Doctors in hospitals just follow rules.Doctors consider various levels of evidence, and are completely aware of the weaknesses of observational studies. They actually conduct randomized experiments, which they consider to be amongst the highest evidence for something. Doctors all know that observational studies are provide weaker evidence.
In fact, you can argue that doctors are even more stringent than economists in how they evaluate studies. The belief amongst doctors is that causality cannot be established outside of randomized trials. Compare that to economists who bloviate about the identification revolution and use appeals to authority or cute arguments to claim causality for their identification approach.This is simply not true. Do you actually talk to doctors?
In all of my medical interactions, they did NOT weigh different evidence. They cited "hospital policy" on every single issue,with no regard to any evidence.Yes, I do talk with many doctors. Maybe the doctors you interact with are not interested in explaining their reasoning to you because you act like a hostile idiot to them.
-
I do not want to get personal here, but there is something I will say as a matter of full disclosure (since that may be driving my comments a wee bit). As a rational decision-maker, I take into account all available information, which includes information about Emily's past ethics as a researcher and the research ethics of her mentors and reviewer network. At a minimum, there is a certain carelessness involved. Given this information, I discount her findings, especially with respect to findings that will impact my unborn child. This is completely consistent with rational behavior. What drives the rationality of posters who do not take into account the information about EOs research ethics, but question the research practices of the medical community? And use this sloppy rationalization to make important life decisions. I would have no hesitation using recommendations formed on the basis of the work another researcher whose research ethics I trust (like Anna in the same department). And I am a theorist who does not have time or training to carefully carry out a complete analysis in this specific field myself.
What are you, a Minnesota Phd? Even theorists take econometrics and have more than enough training to draw some statistical conclusions regarding a t-test in a 5 page medical publication.
-
You're shifting the battleground. Yes, one can question EO's interpretation of the data and research ethics based on her history. But that is basically orthogonal to the question of how much faith to place in doctor's "folk" wisdom. There are a lot of a lot of sound reasons to place less than full confidence in this (based on the direct experiences of many, data, the excessive reliance of most doctors on arbitrary cutoffs developed by non-research organizations for diagnosis and treatment, and common sense).
At this point in my life, my confidence in a doctor is pretty much inversely correlated with their own confidence about their "folk" knowledge. I think there is clearly room for an Oster-like figure to add value by performing a hard-headed analysis of the available research data from an economist's perspective. (See also Cochrane Reviews). Perhaps she is not the "hero" we'd pick, but she's the one we've got right now.
I do not want to get personal here, but there is something I will say as a matter of full disclosure (since that may be driving my comments a wee bit). As a rational decision-maker, I take into account all available information, which includes information about Emily's past ethics as a researcher and the research ethics of her mentors and reviewer network. At a minimum, there is a certain carelessness involved. Given this information, I discount her findings, especially with respect to findings that will impact my unborn child. This is completely consistent with rational behavior. What drives the rationality of posters who do not take into account the information about EOs research ethics, but question the research practices of the medical community? And use this sloppy rationalization to make important life decisions. I would have no hesitation using recommendations formed on the basis of the work another researcher whose research ethics I trust (like Anna in the same department). And I am a theorist who does not have time or training to carefully carry out a complete analysis in this specific field myself.
-
I will further add that the perspective of "costs and benefits" is severely lacking in medical recommendations, particularly with respect to new parents. There are literally countless examples.
"Rooming in" increasing the chance of dropping or crushing your child in the hospital. Mandatory breastfeeding significantly exacerbating the stress of new mothers (who are very unequal in their ability to produce milk), with negative consequences on their temperament and ability to bond with their children. Same goes for the recommendation to sleep in the same room with your infant for *a year*. The medical professional overtly treats many costly decisions as "free," with real psychological and medical consequences for parents and children. Oster is an imperfect messenger, but IMO is bringing some much needed perspective to a bats**t medical community.
You're shifting the battleground. Yes, one can question EO's interpretation of the data and research ethics based on her history. But that is basically orthogonal to the question of how much faith to place in doctor's "folk" wisdom. There are a lot of a lot of sound reasons to place less than full confidence in this (based on the direct experiences of many, data, the excessive reliance of most doctors on arbitrary cutoffs developed by non-research organizations for diagnosis and treatment, and common sense).
At this point in my life, my confidence in a doctor is pretty much inversely correlated with their own confidence about their "folk" knowledge. I think there is clearly room for an Oster-like figure to add value by performing a hard-headed analysis of the available research data from an economist's perspective. (See also Cochrane Reviews). Perhaps she is not the "hero" we'd pick, but she's the one we've got right now.I do not want to get personal here, but there is something I will say as a matter of full disclosure (since that may be driving my comments a wee bit). As a rational decision-maker, I take into account all available information, which includes information about Emily's past ethics as a researcher and the research ethics of her mentors and reviewer network. At a minimum, there is a certain carelessness involved. Given this information, I discount her findings, especially with respect to findings that will impact my unborn child. This is completely consistent with rational behavior. What drives the rationality of posters who do not take into account the information about EOs research ethics, but question the research practices of the medical community? And use this sloppy rationalization to make important life decisions. I would have no hesitation using recommendations formed on the basis of the work another researcher whose research ethics I trust (like Anna in the same department). And I am a theorist who does not have time or training to carefully carry out a complete analysis in this specific field myself.
-
I can do that with one paper, or even two. But forming an opinion would mean careful study at a deeper level and engaging with the literature. I do not have time for that at this stage of my career and life.
What are you, a Minnesota Phd? Even theorists take econometrics and have more than enough training to draw some statistical conclusions regarding a t-test in a 5 page medical publication.
-
What Dumbo doctors apologists post here?
Med research is entirely phacking on observational studies.
Doctors in hospitals just follow rules.Doctors consider various levels of evidence, and are completely aware of the weaknesses of observational studies. They actually conduct randomized experiments, which they consider to be amongst the highest evidence for something. Doctors all know that observational studies are provide weaker evidence.
In fact, you can argue that doctors are even more stringent than economists in how they evaluate studies. The belief amongst doctors is that causality cannot be established outside of randomized trials. Compare that to economists who bloviate about the identification revolution and use appeals to authority or cute arguments to claim causality for their identification approach.This is simply not true. Do you actually talk to doctors?
In all of my medical interactions, they did NOT weigh different evidence. They cited "hospital policy" on every single issue,with no regard to any evidence.Yes, I do talk with many doctors. Maybe the doctors you interact with are not interested in explaining their reasoning to you because you act like a hostile idiot to them.
99% of doctors in the US don't have reasoning beyond "it's what we do"
-
The cost aspect of this "particular question" - namely giving up drinking for for a few months, is negligible. As I mentioned in one of my previous posts, other than an alcoholic I would expect this cost to be casually borne by any normal partner.
I will further add that the perspective of "costs and benefits" is severely lacking in medical recommendations, particularly with respect to new parents. There are literally countless examples.
"Rooming in" increasing the chance of dropping or crushing your child in the hospital. Mandatory breastfeeding significantly exacerbating the stress of new mothers (who are very unequal in their ability to produce milk), with negative consequences on their temperament and ability to bond with their children. Same goes for the recommendation to sleep in the same room with your infant for *a year*. The medical professional overtly treats many costly decisions as "free," with real psychological and medical consequences for parents and children. Oster is an imperfect messenger, but IMO is bringing some much needed perspective to a bats**t medical community.You're shifting the battleground. Yes, one can question EO's interpretation of the data and research ethics based on her history. But that is basically orthogonal to the question of how much faith to place in doctor's "folk" wisdom. There are a lot of a lot of sound reasons to place less than full confidence in this (based on the direct experiences of many, data, the excessive reliance of most doctors on arbitrary cutoffs developed by non-research organizations for diagnosis and treatment, and common sense).
At this point in my life, my confidence in a doctor is pretty much inversely correlated with their own confidence about their "folk" knowledge. I think there is clearly room for an Oster-like figure to add value by performing a hard-headed analysis of the available research data from an economist's perspective. (See also Cochrane Reviews). Perhaps she is not the "hero" we'd pick, but she's the one we've got right now.I do not want to get personal here, but there is something I will say as a matter of full disclosure (since that may be driving my comments a wee bit). As a rational decision-maker, I take into account all available information, which includes information about Emily's past ethics as a researcher and the research ethics of her mentors and reviewer network. At a minimum, there is a certain carelessness involved. Given this information, I discount her findings, especially with respect to findings that will impact my unborn child. This is completely consistent with rational behavior. What drives the rationality of posters who do not take into account the information about EOs research ethics, but question the research practices of the medical community? And use this sloppy rationalization to make important life decisions. I would have no hesitation using recommendations formed on the basis of the work another researcher whose research ethics I trust (like Anna in the same department). And I am a theorist who does not have time or training to carefully carry out a complete analysis in this specific field myself.