I recently wondered how the Soviet healthcare system worked. Initially, I thought, based on broad ideas about the USSR, that it wouldn’t work very well. After a literature review, I confirmed that it was the case.
But would it have been epistemically responsible to proclaim that it was awful without having done that literature review? Probably yes, but without much confidence. Before the review, I just had my opinion, based on background knowledge about how central planning works. Perhaps I was wrong on this specific issue. It wouldn’t have been the first time when I say that something is the case based on background knowledge, and end up being wrong. That’s why one shouldn’t hold those pre-literature review opinions too strongly.
So do we need to do literature reviews ourselves in order to believe anything? That would be too much. Imagine having to spend your life reading the primary, secondary, and journal sources for astrology, homeopathy, conspiracy theories, alchemy, shamanism, or the thousands of religions that are around. All that in order to not believe, or criticise those things. You wouldn’t have time for doing other things.
Another alternative is to see if there is expert consensus on that issue: if there is a large group of people who have studied the issue, and they have come to a conclusion, that counts as evidence that such a thing is indeed the case. This evidence can be even stronger than your peer review: if you review the literature of a field, and come to conclusions opposite to that of the experts in that field, you have to find the reason for the discrepancy. If you can’t, you probably have to stick with the consensus opinion.
For homeopathy there are some papers debunking it (NHMRC, 2015, or Hahn 2013, or Shang et al. 2005, or Ernst 2002) but then, there are also papers saying that it works published in journals (Mathie, 2014). In this cases, the reasonable thing to do is a) Go with the latest findings b) Go with the consensus and c) See if the pro-homeopathy findings have been criticised, or if the anti-homeopathy papers have gotten a credible answer from homeopaths.
So we can go to the papers that cite that one and read Mathie, 2015 published in the journal Homeopathy, which happens to be indexed in the JCR. He says that there is weak evidence for positive effects of homeopathy after reviewing the literature. The NHMRC study was also replied to by homeopaths pointing out to the literature itself.
As for the mechanism (diluting repeatedly some substance), homeopaths also have their papers. So when you say “But diluting something will make it vanish, in the end it’s just water” will be answered with “Sure that sounds plausible, but turns out it isn’t. Here’s evidence that it does work”. Didn’t expect that? Thought homeopaths didn’t have cutting edge research on their side? And many of them have not been replied to by other scientists. But see here for one such reply. See Grimes (2012) for how it is physically implausible
I think homeopathy does not work, and the consensus of medical scientists thinks the same. The mechanism they propose for it to work does not make physical sense. For me to believe that it works, they would need to give me very strong evidence that it works. Since the critics say it doesn’t work, and the homeopaths themselves say their evidence is, at best, weak, the plausible conclusion to draw is that, indeed, it doesn’t work. Perhaps in the future it will be shown that it works, but it is unlikely.
Should we consider homeopathy to be pseudoscientific then? This is an amusing question. Seemingly, homeopaths are doing science to try to prove homeopathy works. They are doing RCTs, publishing papers, answering criticisms, and trying to study the properties of water to see how their proposed mechanism might work. Their methodology is most probably flawed, but so is the methodology of many papers in other fields. In the wiki article there are some explanations for why they may be finding positive results sometimes. Here there are some definitions of pseudoscience. Homeopathy is falsifiable, for one, so for Karl Popper, homeopathy would be a science. Homeopathy also made predictions that, according to their followers, came to be true (according to their research): they supposedly found evidence for mechanisms in water.
I don’t want to go deep into their literature to assess it and check if some symptoms of pseudoscience (as explained in the wiki article) are present, because I think it would be worthless.
To sum up, we have:
- For homeopathy: Some weak meta-analytical evidence for the principles and individual illnesses, endorsement from a small group (homeopaths)
- Against homeopathy: Violation of the laws of physics, the consensus of medical scientists, several meta-analysis finding no effects.Most of the evidence for homeopathy comes from homeopaths.
With this quick assessment, without having read more than one or two papers of a vast literature, I can safely dismiss homeopathy, in a way that I think, homeopaths themselves would understand, instead of getting mad at me for my rejection. Anyone in my position and with the information I have would have to conclude this. Homeopaths may think that in the future they will find stronger evidence for it, and so they will keep trying. But until then, they will have to understand that it is accurate for an average person, or someone who has done a bit more of work, like me, to describe homeopathy as wrong. Depending on how charitable you feel, you may call it pseudoscience too. I haven’t done a study of the concept of pseudoscience. What if something is not a science? Can we really find tight criteria for what it is and isn’t a science? I care more about whether something is right or wrong.
Some heuristics to safely dismiss and accept things with little information
Start with simple procedures, move on to more time-consuming ones if you think you need more confidence in your belief.
- See if it makes sense. This usually works most of the time. This consists of checking an issue against your background beliefs, among which, hopefully, is an understanding the laws of physics.
- See what the consensus on that issue is
- This can be done by finding statements from important institutions
- Or by reading meta-analysis
- Read critiques of the consensus positions. Read critiques of those critiques. Read critiques of the critiques of the critiques, and so on until you reach the current year. Google Scholar can help you to do that.
- Do a full-scale literature review, read most of the papers that have been written on the issue
- And finally, assess the literature yourself, looking for methodological flaws.
Understand why people disagree with you
After looking for resources on homeopathy, I think it is easy to understand why there are people who believe in it.Some people react to this fact by mentally facepalming, or getting angry with them, or insulting them (“How can you be so stupid! It is obvious that it doesn’t work!”). I react by understanding the sources of the beliefs of the believer in homeopathy. I infer there are two types of people who believe in it: regular people who buy homeopathy, and homeopaths themselves.
For the first group, they probably came to believe in it because some people in their surroundings tried it, and said it work, and perhaps they looked for information on the net and found homeopathic websites and they told them it worked. Or maybe, they just gave it a try, and they got better, and they didn’t know about the post hoc fallacy.
For the second group, they have read the literature, and know what is going on. They have gone to levels 4 or 5 in my scale of heuristics, and still believe in homeopathy. This probably means that they began with a strong or moderately strong belief that homeopathy worked, and then they haven’t found sufficient proof to abandon that belief. They should stop just adjusting their beliefs on the basis on new evidence and review whether their initial opinion was correct at all. Studies will keep coming, and in the end, if they are honest, and science points that way, they will stop believing in homeopathy.
If you want to engage with homeopaths -I personally find it a waste of time, but maybe you don’t-make sure to understand them, and see what sources of evidence they may have, and explain why they don’t work, rather than blaming people for their stupidity. People don’t like being called stupid, and it may backfire: it may make believe even more in homeopathy.
There ain’t such thing as a free lunch. Acquiring information has opportunity costs. There is a tradeoff between certainty and amount of work: the more work you do to acquire information, the more certain you can be. So it is useful, or even, necessary, to have heuristics to quickly reach high enough confidence levels on the truth of a given belief with small amounts of information. They are heuristics, and may fail, so be prepared to change your mind when new evidence comes in, or when you reassess evidence you already knew. In this post, I offer some heuristics to do this.
Once a belief has been dismissed or accepted, you can stay with that dismissal or acceptance, and if you want, occasionally check new evidence to see if it support your initial assessment. If you happen to hold a belief that, for most reasonable people, is false, the burden of proof is on you to produce a low-acquisition-cost response to convince them that the world is wrong and you are right. This is basically doing a level-4/5 review of the evidence, and being so charitable that your your critics get the impression that they are being understood, and their criticisms are being addressed.