I really appreciate this newsletter. A few months ago I started bingeing Maintenance Phase episodes. Their perspective fascinated me, and I found their (apparant) ability to produce endless examples of blatant bias and dysfunction in the scientific research compelling. Most of all, I trusted them because they were speaking out against the ugly and dehumanizing way our culture treats and talks about obese people.
But for me, the fly in the ointment was when they talked about how the laws of thermodynamics don’t apply to the human body. It was jarring and surreal to hear people I trusted so confidently argue a point that made such little sense. If they were this wrong about something so basic, what else could they be wrong about? I stopped automatically trusting what they said, and eventually I came across your writing. Thank you for all the work you do to show the full and alarming extent of the misinformation of these podcasts. It’s really helpful for people like me who don’t have a scientific background.
One of the things I appreciate most about your writing is how you’ve stood up for science in spite of it’s flaws. I think a big reason MP is so popular is that there are real and major problems in science and science communication which many feel intimidated into not freely acknowledging. It’s gratifying and validating to listen to a knowledgable person send up the scientific establishment for its hubris. The problem is that this instinct to pile on to science can open the door for all sorts of misinformation. Your work has been a good counter balance for my bias in this direction.
For this reason I would be interested to hear your thoughts on the Substack Newsletter ‘Experimental History.’ EH is one of the best Substacks I’ve come across. Adam Mastroianni is a wickedly talented writer and seems to have a strong scientific background. I haven’t come across any glaring logical or factual mistakes in his critiques of the scientific establishment, but I can’t help but wonder if science also deserves someone to stand up for it here too.
Thank you so much for this thoughtful response! I agree with everything that you have said - especially that today, when nuance is hard to find in online discourse, any criticism of the scientific establishment tends to snowball into anti-science rhetoric and misinformation, especially in non-scientific circles. I don't know what the solution, but I think that when we dunk on aspects of science (like peer review, for example), we need to include discussions of what the ideal scenario would look like instead of just purely dunking. Every system is flawed, but that doesn't mean that they are all irreparable or that we should just disregard them entirely (some we should, for sure, but not all).
I am a fan of Adam Mastroianni and I also think that the social sciences are very different from the biological/natural sciences, and some of his critiques are really only applicable to the social sciences. This nuance is lost a bit in his writings, because he often addresses them as issues with science as a whole. I think he has a tendency to make broader statements than I feel comfortable making about the scientific establishment as a whole, but then again, that's what gets readership these days! I agree it would be nice to have some counter to his narrative to balance things out.
I admit that this is a bit of an extreme stance, and I’m not sure you’d agree, but I can’t help but think that the goal of MP is there to discredit public health as a concept and perhaps even private health. It just seems like all the “mistakes” and “inaccuracies” point to one conclusion: that positive health changes are not possible for the individual nor the collective
Hi Peter! I have come to (cautiously) think this, as well. It's troubling to see a podcast have this many "mistakes" that all trend in the same direction. The podcast has an undertone of anti-establishment sentiment (which I get, in the sense that anti-fat bias is deeply entrenched in "establishment" culture), but it borders pretty dangerously on the same kind of anti-public health rhetoric that spirals into distrust of science in general. That's what is most alarming to me.
There seem to be a few podcasts/creators going down the 'health is a social construct' and 'lifestyle doesn't impact physiology' path. Even if that's not MP's explicit intention, it does feel like they're feeding into those ideas.
First, a confession: I’ve never listened to a single episode of MP, and only once, long ago caught some portion of “You’re Wrong About” before dipping about based on the level of glib, superficial, and self congratulatory that Hobbes brought to the table, which was jarring even though I had no particular prior knowledge of the subject (not that I even remember what that particular episode was about).
With that confession out of the way, just reading through your excellent critique of this episode made me want to pull my hair out, particularly the hosts stunning inability (unwillingness?) to be able to distinguish between scientific consensus, well validated scientific findings, and actual public health communiques/recommendations vs whatever pop “science” trash floated across their personal radars and outright lunatics.
It’s not to say that that first groups is beyond reproach (hell, I’m in the health policy sphere, and like 90% of my time is devoted to very specific criticism of some aspect of them)…but to just drop Joseph fucking Mercola into an episode about COVID scientific communication without putting into context that he’s an absolute lunatic who has been King of the Quacks for going on two decades…my god, the level of hubris of these two and harm they are doing to their audience is just absolutely shocking.
You describe Michael's schtick perfectly haha. Thank you for reading. You hit the nail on the head! MP very often conflates quacks with actual science. It's bizarre. That's why I struggle to figure out exactly what the point of the podcast is. I know what they SAY the point is, and what fans believe the point is, which is to raise awareness of anti-fat bias and to push the message that fat-phobia is toxic and harmful. But the actual content of the pod really doesn't do that at all. It's very perplexing to me!
I’m very grateful that you’re doing this fact-checking! However at that point I feel stupid believing you, because ofc I take your words without reading papers you link - but I don’t have scientific literacy nor time for that. Like most people. That’s why I rely on people doing science communication.
Yeah I was fan of MP and Michael Hobbes other work. Now I feel very angry that they violated my trust. And at activists in general. In retrospect it’s kinda logical that I shouldn’t trust activists, because activists by definition are to push their agenda, not to carefully communicate sometimes uncomfortable truth.
Thanks for reading and commenting! It is so hard these days to know who to trust. People have asked me why they should trust me over MP. I'm just some rando on the internet! All I can say is that I hope they see that it's not a matter of trusting me, because I can provide citations (whether or not you read them!) and I am also a trained scientist. I'm not infallible, but I do respond to feedback when readers think I'm off the mark.
You raise an important point, which is that we just simply don't have the time to fact-check every piece of news we consume. It's really disillusioning to find out that people we trusted to provide us with reliable information are skewing or misrepresenting facts! I know what that feels like!
It’s funny, I was just listening to that episode of Iron Culture this morning. Do you listen to them regularly? Would you say they have more scientific accuracy than MP? I assume they (and similar/overlapping podcasts like Stronger by Science) are solid based on the level of detail they go into and bent toward epistemic humility, but I’m not an expert.
Regardless, thank you for this series on MP. I’ve enjoyed reading it and learned a lot from it!
Yes! Iron Culture is great - I was actually just introduced to it recently. Eric Helms is a fantastic resource. Stronger by Science is also a great podcast. I have yet to hear them say anything that I think is problematic. They are very diligent about their research, and they are all scientists, so they know what they are talking about! Very different from MP.
Seconding that Iron Culture has leagues more credibility than MP, Eric Helms has a relevant PhD and is a very well respected scientist in his field. For health/nutrition stuff I'd also recommend the podcast Sigma Nutrition Radio. The hosts have relevant degrees, qualified guests, and someone on staff with the job title "research communication officer".
I am a longtime fan of MP and grounding my understanding of the world in good science is very important to me. I have a BA and took a lot of science as an undergrad, but would still consider myself a lay-person. I am really troubled by the idea that a podcast I have looked to for so long to give me accurate information may not be holding themselves to the standard I thought they were.
I'm really intrigued by your substack and am grateful to you for putting in the work to put it together. One thing that would make it easier for me to understand would be if you actually had three sections for each point: MP transcript, your response, and a correction. The correction would be you re-writing the MP transcript in a way that lacks inaccuracies. So if the transcript says "The study says 30% of people are going to get diabetes in their lifetime" and your research found that the study was a prediction of Americans, then your this section would say "The study used a predictive model to estimate that 30% of Americans will get diabetes in their lifetime."
Seeing your correction formatted in the same way that MP has phrased things would really clarify for me what the distinction is you are trying to make. In one of the previous episodes they said "the fattest 15% of people would be considered overweight" and you wrote that the cutoff points "were based on the 85th percentile of BMI for men and women aged 20-29." I spent a few minutes looking at the words "15%" and "85th percentile" trying to understand why those are not just two different ways of saying the same thing. After awhile I realized the distinction you were making was the difference between "of people" aka the whole population and "men and women aged 20-29." So in the third correction section you would change the excerpt to say "In 1995 BMI cutoffs were established that defined overweight as people weighing equal to more than that fattest 15% of people in their 20's." Having these two things juxtaposed would make your corrections so much clearer to me.
Hi Emily! This is excellent feedback. Thank you very much! I can see how that would be more accessible than the format I have been using thus far. I will incorporate that into my next posts, for sure. Again, really appreciate the constructive feedback and very happy to do whatever I can to make these posts more approachable! :)
Hello! I binge read this substack and I really appreciate the work you’re doing! I used to enjoy Michael Hobbes and his podcast “universe” but I started side eying him when I listened to an early You’re Wrong About episode with a friend and we realized that some of the research he cited was super off and it just went…unquestioned? I want to say it was the Ford Pinto or Challenger episode and my friend is an engineer, so they were able to point out the flaws in the episode much better than I could. It kind of just made us reevaluate his work. Obviously he’s no longer on You’re Wrong About but it used to drive my friend and I crazy because it felt like so much of his stuff went unquestioned because he read it out in an authoritative way. I hate to drop him to these levels, but the way his fanbase is forming it really feels like he’s becoming Jordan Peterson/Joe Rogan levels of revered. And like them, he does bring up good points on occasion, but also like them he has a fan base that believes him unquestionably and will bite your head off if you dare even mildly critique him. I don’t know, I feel like he’s on the path to potentially doing a lot of harm (as seen by this fact check specifically!) and I wish more people would push back on him. The tl;dr is that I love this substack and I hope you keep up the good work!
Hi Megan! Sorry for the delayed response! Thank you so much for reading and for commenting. I feel exactly the same way that you do. He has a very cult-like following and people are adamant that he's a fabulous researcher when there is very clear evidence that he's not. His whole schtick is just pandering to an audience that agrees with him. A lot of his fans think I'm just doing this because I have beef with him or because I'm a fatphobe, but it's actually just what you said - I think he's on the path to doing a lot of harm, if he hasn't already. Anyway, I had to take a bit of a hiatus because my actual job got busy but I'm working on another post now so hopefully it will be up in the next couple of weeks!
I so appreciate this substack! I was a regular supporter of MP initially, but after taking a few data analysis courses in grad school (I have a public policy masters) I started to notice some loose threads. For me the big one was their misuse of "statistically significant" in multiple episodes; if they don't understand this pretty important aspect of understanding data analysis, what else are they getting wrong? I sorta stopped listening so much after that. I found this after my spouse put on the "Trouble with Calories" episode the other day and I couldn't put a finger on specifically what the issues were with their discussion. I'm not a scientist (my policy focus is on ASL interpreting and disability policy) so finding this substack has been almost a relief to find- finally someone can articulate the stuff that felt "off" to me but that I didn't have the background to understand why. Thanks for this work!
Hi! Thanks for reading and for the comment! Policy work seems so daunting to me. And thankless! So here, please take this thank you from me for doing the hard work - especially in such an important area!
You hit the nail on the head. It would be so easy for them to just educate themselves a bit better or have some humility and invite on an expert, but instead they are spreading misinformation that is just as bad as what they are trying to "debunk."
I was referred to this substack by a friend who knows I always enjoy a good fact check. I listen to Maintenance Phase from time to time and find it an interesting point of view but certainly not the last word on public health or my personal health care decisions. The podcasters clearly have their own angle they're coming from, and sometimes that meshes with my understanding of the science and sometimes it doesn't. (As background about me, I studied health science and then for a time was a health educator for a pediatric hospital. In case it doesn't go without saying, I ♥️ evidence-based medicine.)
I agree that this was not a great episode. It felt scattered and Michael was touching on too many things to do any of them justice, and often glossed over and clumsily or incorrectly summarized important information because he was trying to cover too many things at once. I particularly appreciated your dive into the ACE inhibitor correspondence.
Still, in some parts of your post, I feel like we didn't listen to the same episode. For example, I never understood Michael to be citing Mercola or Todaro as mainstream medical experts--the podcast was about conspiracies, and he gave them as examples of conspiracists, and how social media and journalism picked up on these conspiracies and spread them. Also, when Michael said veterinary ivermectin "appears" to have the same formulation as human ivermectin, I took that to mean that it *doesn't*, but people think it does. I get that what he said can be interpreted multiple ways, and maybe my interpretation was based on prior knowledge. And if he had said what he meant clearly, it wouldn't have been able to be interpreted in multiple ways. So that's a problem. But the problem is his lack of clarity. Lack of clarity is bad because it can lead people to misunderstand basic science. A fact check should include the clarification that human and veterinary ivermectin are different. But I wouldn't ordinarily expect a fact check to include assumptions about what Michael knows.
I guess I'm just confused about whether this substack is simply for fact checking, or also about looking for fault in these podcasters. Maybe it's here to do both, and my friend misrepresented the purpose of this substack to me. If that's the case, the problem is my expectations when I approached the blog post, not the things I point out here.
In any case, thanks for the research you did and for sharing it with us here. Even if I nitpick with the way the show is interpreted, the facts you present are very helpful.
Hi! Thanks for commenting and I don't think you're nitpicking - I appreciate the constructive feedback. And sorry for the delayed response. :)
I think you have a good point that I listen to these episodes already on the defensive for what they are misrepresenting and getting wrong about science and public health, so I have a much less charitable interpretation of what they say. That's on me and I appreciate the call out. I have been swamped with work so haven't had time to do another fact check recently, but I am going to work on highlighting the larger issues and giving them more benefit of the doubt when it's not entirely clear what they mean (or maybe just saying I'm not sure and acknowledging I could be misinterpreting). I agree the lack of clarity is an issue in and of itself and I also can see that I overstep sometimes and comment on Michael's knowledge vs just the facts themselves. Thanks for pointing that out.
I started doing these fact checks because a friend asked me to point out the errors in the Ozempic episode and then it kind of spiraled from there. My goal really is to fact check, but I won't lie that I also feel frustration about Michael and Aubrey and think that what they are doing is irresponsible journalism. And that boils over into my fact checks sometimes, which it shouldn't. And I'm really going to work on that going forward (when I have time to do another one hah!). Thanks so much, again, for reading and for commenting. I appreciate the feedback.
Thanks for your response! I really appreciate the work you are doing and look forward to reading more of your previous posts. I definitely understand your perspective--for some reason Michael doesn't get on my goat, but I have other podcasters and journalists that I am not so charitable toward. You have spent much more time and energy deep diving into the facts on these episodes, so I imagine that contributes to frustration because each time they say something questionable or non factual, that is more work for you.
Thanks for the kind words. It is funny which things drive us crazy and which don't!
I think you're right that being so deep into the content makes me extra frustrated. I do end up editing and refining the content to take out the things that I recognize upon re-reading are too nitpicky (obviously I still have some work to do there!) and I have a friend who helps who is a Certified Strength and Condition Coach so she knows quite a lot about nutrition and physiology, too. But if I want to be an effective communicator about this, I need to maintain objectivity so that people don't think I just have an agenda to take down Michael and Aubrey (which some people definitely think is my overarching plan). So thanks again for your comments!
Hi! Thank you for all the work you do. As someone who works in science it can be really frustrating to see people misrepresent papers, and it's been even more frustrating to learn that MP has been also doing that (guess the lesson learned is to fact check). I'm looking forward to reading your other substack posts.
You have a link in this post talking about an unreputable journal that pushes papers out quickly, but the link you shared is broken. What journal(s) were you talking about? I ask because I've been researching supplements and adaptogens and have struggled to determine which journals are actually reputable. I know the bigger ones that are more well-known, but it's tricky with the smaller journals that aren't as well known.
Hi Trinity! Thanks for the reading and thanks for the comment. I checked and the link doesn't appear to be broken for me. Are you talking about the sentence that says, "A few wrong things here. First, the Egyptian study was a preprint. The Brazilian study reported a 92% risk of death. It was published in a journal that is very controversial in the scientific community because it has “expedited” peer review." The link is here: https://statmodeling.stat.columbia.edu/2023/08/24/cureus-its-reviewing-and-its-scholarly-impact-quotient/
The journal is called Cureus! There is also Beall's list of Predatory Journals which is helpful (https://beallslist.net/) but has not been updated in several years since he was essentially forced to stop.
I had such high hopes for them after they mentioned they got Health Nerd to fact-check their episode! I'd be interested to hear his side of things as to what they asked him to fact check and why there are still so many inaccuracies because I really like his writing.
I detest MP's armchair expertise and how confidently incorrect they are about science. Such a shame that they won't just get an expert on to discuss these incredibly complex topics! It's exhausting.
I felt the same and was similarly disappointed. I actually wrote to Health Nerd on Substack to hear how they had contacted him and what the experience was like. He told me, "Michael contacted me because they used a lot of my work in the episode, such as our investigation into ivermectin fraud, and several retracted vitamin D studies that I'd worked on." He also told me, "But in general I think they're pretty good with the evidence", but amended that in later messages when I pushed back to say, "I think they tend to be reasonably good in terms of evidence when it's not related to whether fat people should lose weight, but you're right that they otherwise often make quite a few errors." Anyway, I'd like to charitably think that he only was asked to fact-check the things that were directly related to his posts, but even so, there are some things that are completely wrong in those sections. I'm left thinking that maybe his version of fact-checking differs from mine haha. He also says he has a PhD in "diabetes epidemiology" but his university (Wollongong) doesn't actually have any programs in epidemiology, so I'm left just confused in general. There are so many good experts they could have on, it just baffles me that they don't want to do that.
That's an interesting reply from him! I suppose he was being generous with his initial assessment, but I'm happy to hear he recognizes they make numerous errors. It would be soooooo easy to avoid this just by getting qualified people on the podcast! It could be a great podcast highlighting the nuances of health, wellness, and weight loss. But to me it seems that they've weaponized the nuance, and their underlying messaging has devolved into something like, "Weight loss is difficult, so don't bother."
Yes exactly! And yet the podcast is also staunchly anti these new weight loss drugs which can help people lose weight (yes, it has problems, both side effects and socioeconomic ramifications). It's getting hard to figure out exactly what their central values are, because they have such conflicting messages across episodes. I think they have such an incredible opportunity to highlight researchers who are doing important work in this space and/or nutrition/health/fitness experts who are promoting holistic, weight-neutral practices that are evidence-based. But instead it is just a dunk fest.
The dunking kills me because much of the time, its based on a bunch of strawman arguments or a misunderstanding of the way research works. Like when they talk about the side effects of Ozempic (all drugs can cause side effects!!!), or the efficacy-effectiveness gap because they don't understand the difference between a RCT and observational study. It's not that they've necessarily lied in these two examples, but they also haven't told the full picture because they don't understand it.
You articulated my feelings perfectly!! In order to critically appraise something, you have to understand it really well. We can definitely have opinions about things without being experts - goodness knows I sure do - but dissecting something in a public forum requires a special depth of knowledge. I found a NYT article from 2000 titled, "Suddenly, Everybody's an Expert" (https://www.nytimes.com/2000/02/03/technology/suddenly-everybody-s-an-expert.html) and oh boy have things only gotten worse since then. To think we thought this was an issue in 2000!
Yes! I think things have gotten so much worse since that NYT article because now that everyone has a platform, bullshit proliferates so much more quickly these days.
I also catch myself offering opinions on things I don't understand very well, but I try to remember to add qualifiers so I'm not guilty of epistemic trespassing (really enjoy Neil Levy's thoughts on this here: https://link.springer.com/article/10.1007/s10670-024-00794-8). He's also recently spoken about a type of grandstanding that he calls" intellectual virtue signaling", which is also worth a read (https://doi.org/10.5406/21521123.60.3.07).
I really appreciate this newsletter. A few months ago I started bingeing Maintenance Phase episodes. Their perspective fascinated me, and I found their (apparant) ability to produce endless examples of blatant bias and dysfunction in the scientific research compelling. Most of all, I trusted them because they were speaking out against the ugly and dehumanizing way our culture treats and talks about obese people.
But for me, the fly in the ointment was when they talked about how the laws of thermodynamics don’t apply to the human body. It was jarring and surreal to hear people I trusted so confidently argue a point that made such little sense. If they were this wrong about something so basic, what else could they be wrong about? I stopped automatically trusting what they said, and eventually I came across your writing. Thank you for all the work you do to show the full and alarming extent of the misinformation of these podcasts. It’s really helpful for people like me who don’t have a scientific background.
One of the things I appreciate most about your writing is how you’ve stood up for science in spite of it’s flaws. I think a big reason MP is so popular is that there are real and major problems in science and science communication which many feel intimidated into not freely acknowledging. It’s gratifying and validating to listen to a knowledgable person send up the scientific establishment for its hubris. The problem is that this instinct to pile on to science can open the door for all sorts of misinformation. Your work has been a good counter balance for my bias in this direction.
For this reason I would be interested to hear your thoughts on the Substack Newsletter ‘Experimental History.’ EH is one of the best Substacks I’ve come across. Adam Mastroianni is a wickedly talented writer and seems to have a strong scientific background. I haven’t come across any glaring logical or factual mistakes in his critiques of the scientific establishment, but I can’t help but wonder if science also deserves someone to stand up for it here too.
Thank you so much for this thoughtful response! I agree with everything that you have said - especially that today, when nuance is hard to find in online discourse, any criticism of the scientific establishment tends to snowball into anti-science rhetoric and misinformation, especially in non-scientific circles. I don't know what the solution, but I think that when we dunk on aspects of science (like peer review, for example), we need to include discussions of what the ideal scenario would look like instead of just purely dunking. Every system is flawed, but that doesn't mean that they are all irreparable or that we should just disregard them entirely (some we should, for sure, but not all).
I am a fan of Adam Mastroianni and I also think that the social sciences are very different from the biological/natural sciences, and some of his critiques are really only applicable to the social sciences. This nuance is lost a bit in his writings, because he often addresses them as issues with science as a whole. I think he has a tendency to make broader statements than I feel comfortable making about the scientific establishment as a whole, but then again, that's what gets readership these days! I agree it would be nice to have some counter to his narrative to balance things out.
I appreciate the reoccurring bit of Michael being confused by scientific papers being complex and filled with technical jargon and math.
He would benefit so much from just having an expert on!
I admit that this is a bit of an extreme stance, and I’m not sure you’d agree, but I can’t help but think that the goal of MP is there to discredit public health as a concept and perhaps even private health. It just seems like all the “mistakes” and “inaccuracies” point to one conclusion: that positive health changes are not possible for the individual nor the collective
Hi Peter! I have come to (cautiously) think this, as well. It's troubling to see a podcast have this many "mistakes" that all trend in the same direction. The podcast has an undertone of anti-establishment sentiment (which I get, in the sense that anti-fat bias is deeply entrenched in "establishment" culture), but it borders pretty dangerously on the same kind of anti-public health rhetoric that spirals into distrust of science in general. That's what is most alarming to me.
There seem to be a few podcasts/creators going down the 'health is a social construct' and 'lifestyle doesn't impact physiology' path. Even if that's not MP's explicit intention, it does feel like they're feeding into those ideas.
First, a confession: I’ve never listened to a single episode of MP, and only once, long ago caught some portion of “You’re Wrong About” before dipping about based on the level of glib, superficial, and self congratulatory that Hobbes brought to the table, which was jarring even though I had no particular prior knowledge of the subject (not that I even remember what that particular episode was about).
With that confession out of the way, just reading through your excellent critique of this episode made me want to pull my hair out, particularly the hosts stunning inability (unwillingness?) to be able to distinguish between scientific consensus, well validated scientific findings, and actual public health communiques/recommendations vs whatever pop “science” trash floated across their personal radars and outright lunatics.
It’s not to say that that first groups is beyond reproach (hell, I’m in the health policy sphere, and like 90% of my time is devoted to very specific criticism of some aspect of them)…but to just drop Joseph fucking Mercola into an episode about COVID scientific communication without putting into context that he’s an absolute lunatic who has been King of the Quacks for going on two decades…my god, the level of hubris of these two and harm they are doing to their audience is just absolutely shocking.
You describe Michael's schtick perfectly haha. Thank you for reading. You hit the nail on the head! MP very often conflates quacks with actual science. It's bizarre. That's why I struggle to figure out exactly what the point of the podcast is. I know what they SAY the point is, and what fans believe the point is, which is to raise awareness of anti-fat bias and to push the message that fat-phobia is toxic and harmful. But the actual content of the pod really doesn't do that at all. It's very perplexing to me!
I’m very grateful that you’re doing this fact-checking! However at that point I feel stupid believing you, because ofc I take your words without reading papers you link - but I don’t have scientific literacy nor time for that. Like most people. That’s why I rely on people doing science communication.
Yeah I was fan of MP and Michael Hobbes other work. Now I feel very angry that they violated my trust. And at activists in general. In retrospect it’s kinda logical that I shouldn’t trust activists, because activists by definition are to push their agenda, not to carefully communicate sometimes uncomfortable truth.
Thanks for reading and commenting! It is so hard these days to know who to trust. People have asked me why they should trust me over MP. I'm just some rando on the internet! All I can say is that I hope they see that it's not a matter of trusting me, because I can provide citations (whether or not you read them!) and I am also a trained scientist. I'm not infallible, but I do respond to feedback when readers think I'm off the mark.
You raise an important point, which is that we just simply don't have the time to fact-check every piece of news we consume. It's really disillusioning to find out that people we trusted to provide us with reliable information are skewing or misrepresenting facts! I know what that feels like!
It’s funny, I was just listening to that episode of Iron Culture this morning. Do you listen to them regularly? Would you say they have more scientific accuracy than MP? I assume they (and similar/overlapping podcasts like Stronger by Science) are solid based on the level of detail they go into and bent toward epistemic humility, but I’m not an expert.
Regardless, thank you for this series on MP. I’ve enjoyed reading it and learned a lot from it!
Yes! Iron Culture is great - I was actually just introduced to it recently. Eric Helms is a fantastic resource. Stronger by Science is also a great podcast. I have yet to hear them say anything that I think is problematic. They are very diligent about their research, and they are all scientists, so they know what they are talking about! Very different from MP.
Seconding that Iron Culture has leagues more credibility than MP, Eric Helms has a relevant PhD and is a very well respected scientist in his field. For health/nutrition stuff I'd also recommend the podcast Sigma Nutrition Radio. The hosts have relevant degrees, qualified guests, and someone on staff with the job title "research communication officer".
I am a longtime fan of MP and grounding my understanding of the world in good science is very important to me. I have a BA and took a lot of science as an undergrad, but would still consider myself a lay-person. I am really troubled by the idea that a podcast I have looked to for so long to give me accurate information may not be holding themselves to the standard I thought they were.
I'm really intrigued by your substack and am grateful to you for putting in the work to put it together. One thing that would make it easier for me to understand would be if you actually had three sections for each point: MP transcript, your response, and a correction. The correction would be you re-writing the MP transcript in a way that lacks inaccuracies. So if the transcript says "The study says 30% of people are going to get diabetes in their lifetime" and your research found that the study was a prediction of Americans, then your this section would say "The study used a predictive model to estimate that 30% of Americans will get diabetes in their lifetime."
Seeing your correction formatted in the same way that MP has phrased things would really clarify for me what the distinction is you are trying to make. In one of the previous episodes they said "the fattest 15% of people would be considered overweight" and you wrote that the cutoff points "were based on the 85th percentile of BMI for men and women aged 20-29." I spent a few minutes looking at the words "15%" and "85th percentile" trying to understand why those are not just two different ways of saying the same thing. After awhile I realized the distinction you were making was the difference between "of people" aka the whole population and "men and women aged 20-29." So in the third correction section you would change the excerpt to say "In 1995 BMI cutoffs were established that defined overweight as people weighing equal to more than that fattest 15% of people in their 20's." Having these two things juxtaposed would make your corrections so much clearer to me.
Thanks for your work!
Hi Emily! This is excellent feedback. Thank you very much! I can see how that would be more accessible than the format I have been using thus far. I will incorporate that into my next posts, for sure. Again, really appreciate the constructive feedback and very happy to do whatever I can to make these posts more approachable! :)
Hello! I binge read this substack and I really appreciate the work you’re doing! I used to enjoy Michael Hobbes and his podcast “universe” but I started side eying him when I listened to an early You’re Wrong About episode with a friend and we realized that some of the research he cited was super off and it just went…unquestioned? I want to say it was the Ford Pinto or Challenger episode and my friend is an engineer, so they were able to point out the flaws in the episode much better than I could. It kind of just made us reevaluate his work. Obviously he’s no longer on You’re Wrong About but it used to drive my friend and I crazy because it felt like so much of his stuff went unquestioned because he read it out in an authoritative way. I hate to drop him to these levels, but the way his fanbase is forming it really feels like he’s becoming Jordan Peterson/Joe Rogan levels of revered. And like them, he does bring up good points on occasion, but also like them he has a fan base that believes him unquestionably and will bite your head off if you dare even mildly critique him. I don’t know, I feel like he’s on the path to potentially doing a lot of harm (as seen by this fact check specifically!) and I wish more people would push back on him. The tl;dr is that I love this substack and I hope you keep up the good work!
Hi Megan! Sorry for the delayed response! Thank you so much for reading and for commenting. I feel exactly the same way that you do. He has a very cult-like following and people are adamant that he's a fabulous researcher when there is very clear evidence that he's not. His whole schtick is just pandering to an audience that agrees with him. A lot of his fans think I'm just doing this because I have beef with him or because I'm a fatphobe, but it's actually just what you said - I think he's on the path to doing a lot of harm, if he hasn't already. Anyway, I had to take a bit of a hiatus because my actual job got busy but I'm working on another post now so hopefully it will be up in the next couple of weeks!
I so appreciate this substack! I was a regular supporter of MP initially, but after taking a few data analysis courses in grad school (I have a public policy masters) I started to notice some loose threads. For me the big one was their misuse of "statistically significant" in multiple episodes; if they don't understand this pretty important aspect of understanding data analysis, what else are they getting wrong? I sorta stopped listening so much after that. I found this after my spouse put on the "Trouble with Calories" episode the other day and I couldn't put a finger on specifically what the issues were with their discussion. I'm not a scientist (my policy focus is on ASL interpreting and disability policy) so finding this substack has been almost a relief to find- finally someone can articulate the stuff that felt "off" to me but that I didn't have the background to understand why. Thanks for this work!
Hi! Thanks for reading and for the comment! Policy work seems so daunting to me. And thankless! So here, please take this thank you from me for doing the hard work - especially in such an important area!
You hit the nail on the head. It would be so easy for them to just educate themselves a bit better or have some humility and invite on an expert, but instead they are spreading misinformation that is just as bad as what they are trying to "debunk."
I was referred to this substack by a friend who knows I always enjoy a good fact check. I listen to Maintenance Phase from time to time and find it an interesting point of view but certainly not the last word on public health or my personal health care decisions. The podcasters clearly have their own angle they're coming from, and sometimes that meshes with my understanding of the science and sometimes it doesn't. (As background about me, I studied health science and then for a time was a health educator for a pediatric hospital. In case it doesn't go without saying, I ♥️ evidence-based medicine.)
I agree that this was not a great episode. It felt scattered and Michael was touching on too many things to do any of them justice, and often glossed over and clumsily or incorrectly summarized important information because he was trying to cover too many things at once. I particularly appreciated your dive into the ACE inhibitor correspondence.
Still, in some parts of your post, I feel like we didn't listen to the same episode. For example, I never understood Michael to be citing Mercola or Todaro as mainstream medical experts--the podcast was about conspiracies, and he gave them as examples of conspiracists, and how social media and journalism picked up on these conspiracies and spread them. Also, when Michael said veterinary ivermectin "appears" to have the same formulation as human ivermectin, I took that to mean that it *doesn't*, but people think it does. I get that what he said can be interpreted multiple ways, and maybe my interpretation was based on prior knowledge. And if he had said what he meant clearly, it wouldn't have been able to be interpreted in multiple ways. So that's a problem. But the problem is his lack of clarity. Lack of clarity is bad because it can lead people to misunderstand basic science. A fact check should include the clarification that human and veterinary ivermectin are different. But I wouldn't ordinarily expect a fact check to include assumptions about what Michael knows.
I guess I'm just confused about whether this substack is simply for fact checking, or also about looking for fault in these podcasters. Maybe it's here to do both, and my friend misrepresented the purpose of this substack to me. If that's the case, the problem is my expectations when I approached the blog post, not the things I point out here.
In any case, thanks for the research you did and for sharing it with us here. Even if I nitpick with the way the show is interpreted, the facts you present are very helpful.
Hi! Thanks for commenting and I don't think you're nitpicking - I appreciate the constructive feedback. And sorry for the delayed response. :)
I think you have a good point that I listen to these episodes already on the defensive for what they are misrepresenting and getting wrong about science and public health, so I have a much less charitable interpretation of what they say. That's on me and I appreciate the call out. I have been swamped with work so haven't had time to do another fact check recently, but I am going to work on highlighting the larger issues and giving them more benefit of the doubt when it's not entirely clear what they mean (or maybe just saying I'm not sure and acknowledging I could be misinterpreting). I agree the lack of clarity is an issue in and of itself and I also can see that I overstep sometimes and comment on Michael's knowledge vs just the facts themselves. Thanks for pointing that out.
I started doing these fact checks because a friend asked me to point out the errors in the Ozempic episode and then it kind of spiraled from there. My goal really is to fact check, but I won't lie that I also feel frustration about Michael and Aubrey and think that what they are doing is irresponsible journalism. And that boils over into my fact checks sometimes, which it shouldn't. And I'm really going to work on that going forward (when I have time to do another one hah!). Thanks so much, again, for reading and for commenting. I appreciate the feedback.
Thanks for your response! I really appreciate the work you are doing and look forward to reading more of your previous posts. I definitely understand your perspective--for some reason Michael doesn't get on my goat, but I have other podcasters and journalists that I am not so charitable toward. You have spent much more time and energy deep diving into the facts on these episodes, so I imagine that contributes to frustration because each time they say something questionable or non factual, that is more work for you.
At least their schedule has slowed down!
Thanks for the kind words. It is funny which things drive us crazy and which don't!
I think you're right that being so deep into the content makes me extra frustrated. I do end up editing and refining the content to take out the things that I recognize upon re-reading are too nitpicky (obviously I still have some work to do there!) and I have a friend who helps who is a Certified Strength and Condition Coach so she knows quite a lot about nutrition and physiology, too. But if I want to be an effective communicator about this, I need to maintain objectivity so that people don't think I just have an agenda to take down Michael and Aubrey (which some people definitely think is my overarching plan). So thanks again for your comments!
Hi! Thank you for all the work you do. As someone who works in science it can be really frustrating to see people misrepresent papers, and it's been even more frustrating to learn that MP has been also doing that (guess the lesson learned is to fact check). I'm looking forward to reading your other substack posts.
You have a link in this post talking about an unreputable journal that pushes papers out quickly, but the link you shared is broken. What journal(s) were you talking about? I ask because I've been researching supplements and adaptogens and have struggled to determine which journals are actually reputable. I know the bigger ones that are more well-known, but it's tricky with the smaller journals that aren't as well known.
Hi Trinity! Thanks for the reading and thanks for the comment. I checked and the link doesn't appear to be broken for me. Are you talking about the sentence that says, "A few wrong things here. First, the Egyptian study was a preprint. The Brazilian study reported a 92% risk of death. It was published in a journal that is very controversial in the scientific community because it has “expedited” peer review." The link is here: https://statmodeling.stat.columbia.edu/2023/08/24/cureus-its-reviewing-and-its-scholarly-impact-quotient/
The journal is called Cureus! There is also Beall's list of Predatory Journals which is helpful (https://beallslist.net/) but has not been updated in several years since he was essentially forced to stop.
I had such high hopes for them after they mentioned they got Health Nerd to fact-check their episode! I'd be interested to hear his side of things as to what they asked him to fact check and why there are still so many inaccuracies because I really like his writing.
I detest MP's armchair expertise and how confidently incorrect they are about science. Such a shame that they won't just get an expert on to discuss these incredibly complex topics! It's exhausting.
I felt the same and was similarly disappointed. I actually wrote to Health Nerd on Substack to hear how they had contacted him and what the experience was like. He told me, "Michael contacted me because they used a lot of my work in the episode, such as our investigation into ivermectin fraud, and several retracted vitamin D studies that I'd worked on." He also told me, "But in general I think they're pretty good with the evidence", but amended that in later messages when I pushed back to say, "I think they tend to be reasonably good in terms of evidence when it's not related to whether fat people should lose weight, but you're right that they otherwise often make quite a few errors." Anyway, I'd like to charitably think that he only was asked to fact-check the things that were directly related to his posts, but even so, there are some things that are completely wrong in those sections. I'm left thinking that maybe his version of fact-checking differs from mine haha. He also says he has a PhD in "diabetes epidemiology" but his university (Wollongong) doesn't actually have any programs in epidemiology, so I'm left just confused in general. There are so many good experts they could have on, it just baffles me that they don't want to do that.
That's an interesting reply from him! I suppose he was being generous with his initial assessment, but I'm happy to hear he recognizes they make numerous errors. It would be soooooo easy to avoid this just by getting qualified people on the podcast! It could be a great podcast highlighting the nuances of health, wellness, and weight loss. But to me it seems that they've weaponized the nuance, and their underlying messaging has devolved into something like, "Weight loss is difficult, so don't bother."
Yes exactly! And yet the podcast is also staunchly anti these new weight loss drugs which can help people lose weight (yes, it has problems, both side effects and socioeconomic ramifications). It's getting hard to figure out exactly what their central values are, because they have such conflicting messages across episodes. I think they have such an incredible opportunity to highlight researchers who are doing important work in this space and/or nutrition/health/fitness experts who are promoting holistic, weight-neutral practices that are evidence-based. But instead it is just a dunk fest.
The dunking kills me because much of the time, its based on a bunch of strawman arguments or a misunderstanding of the way research works. Like when they talk about the side effects of Ozempic (all drugs can cause side effects!!!), or the efficacy-effectiveness gap because they don't understand the difference between a RCT and observational study. It's not that they've necessarily lied in these two examples, but they also haven't told the full picture because they don't understand it.
You articulated my feelings perfectly!! In order to critically appraise something, you have to understand it really well. We can definitely have opinions about things without being experts - goodness knows I sure do - but dissecting something in a public forum requires a special depth of knowledge. I found a NYT article from 2000 titled, "Suddenly, Everybody's an Expert" (https://www.nytimes.com/2000/02/03/technology/suddenly-everybody-s-an-expert.html) and oh boy have things only gotten worse since then. To think we thought this was an issue in 2000!
Yes! I think things have gotten so much worse since that NYT article because now that everyone has a platform, bullshit proliferates so much more quickly these days.
I also catch myself offering opinions on things I don't understand very well, but I try to remember to add qualifiers so I'm not guilty of epistemic trespassing (really enjoy Neil Levy's thoughts on this here: https://link.springer.com/article/10.1007/s10670-024-00794-8). He's also recently spoken about a type of grandstanding that he calls" intellectual virtue signaling", which is also worth a read (https://doi.org/10.5406/21521123.60.3.07).