Do you know what they call alternative medicine that's proven to work?

When I have a kid, if there's a single thing I can teach them about the world, and if that thing is standard deviation, I will die happy. This is because I believe we have trouble understanding what knowing means.

When we know something (and I mean we know know it), what we do is that we expect something else that is similar in a certain way to behave in a way similar to what we saw before. When we know that rain comes from clouds, we expect seeing clouds whenever there is rain. We know such things because they are true in general or on average. Of course, there is knowledge about specific things, but that is always retrospective, that is always about the past. For example, you may be vary of touching an iron that's switched on because you know it's hot. However, one particular iron may be broken and not produce any heat at all. Once you have examined it, you will know if that is the case. And next time, when seeing that particular iron, you won't expect it to be hot. That's how knowledge works.

And that's how medicines work. When a doctor prescribes some painkillers for you, they do it because they know that usually, on average, it helps people to feel better. How do they know that? When testing the medicine, they must have collected a number of people with some specific symptoms. Some of them felt pretty bad, some relatively OK. Like so:

In cases like this, the graph of the number of people will usually resemble a Gaussian function. Most people will feel bad, quite a few a bit better or worse than that. A small number of people, however, will feel much better or much worse than the rest.

Now, the medicine is administered, the doctors wait a bit, and then ask everyone again how they feel. They find this:

In other words, people on average felt better after receiving a medicine. If the effect (the green arrow) is large enough so that it can't be explained away as an error in measurement, then the medicine is deemed effective. Then we know that the medicine works -- because it works in general.

Notice, however, that the shift towards feeling better can be much-much smaller than the "width" of the original bell-shaped curve. That width, the degree to which people may feel better or worse than the average, is measured by determining the variance or standard deviation. If it's quite big, and in the real world, it can very well be, then there will be people who still won't feel too good, even after swallowing that experimental pill.

And it gets worse.

The medicine works, there's no problem with that. It works because our bodies are sufficiently similar that they react more or less in the same way to the medicine, and, instead of doing completely random things, we, on average, get better. However, this average masks all individual differences. We may have slightly different illnesses, underlying conditions, or genetics. All these change how we react to some medicine, and if we trace the individuals' reaction, the picture gets real messy:

Quite a few people (see the green arrows), as expected, got better. But some people didn't feel any change (grey arrows) or actually got worse (red arrows). Was that because of the medicine? Who knows. Further testing may reveal that they actually had another illness than the rest. Or they were allergic to some component of the medicine. Or they were just unlucky. If there are quite a few people who react badly to a medicine, if it has a particularly frequent and bad side-effect, it of course should be banned or investigated further so that we'd know when it is absolutely safe to use it.

Also, during the course of the treatment, the doctor may discover that a particular patient reacts badly to some medicine, and they'd switch to another. Because then they'd know that; they would have learned that about the patient (much like about the iron in our first example). But when they prescribed the first medicine, they did the best they could.

It is also important to realise that what happens to individuals is hardly ever what happens to the average. Whatever medicine is prescribed (or not prescribed), there will always be that Aunt Margaret or Uncle Sydney who died five days later or miraculously recovered even though they'd forgotten to take the medicine at all. That does not mean that the medicine does not work, or that we shouldn't expect it to work next time. Nor does it mean that we can do what Neighbour Nelly did (have a glass of wine and a cold bath with vinegar) and expect the same outcome. We may get better like them, or we may not.

Knowledge is on average. Individual stories are great to listen to or turn into movies, but until something happens on a large scale, it is not something we can rely on. It's not something we know.

Standard deviation is also important to remember when one encounters stereotypes. Let's say that there are more crimes committed in town A than in town B. Naturally, people hoping for a better future in town A start migrating to town B. However, they are constantly turned down when applying for jobs, and native B's, among each other, just refer to the newcomers as "You know he's an A. You wouldn't want him near your house."

So, according to the example, somehow we know that A's commit more crimes than B's. Fine -- but what does that mean? This:

It means that on average, A's commit more crimes. However, because the deviation is so large, there are plenty of A's who are law-abiding citizens, many even more so than many B's. Similarly, town B is not devoid of criminal elements at all -- and it wasn't even before the migration.

And these statistics probably only scratch the surface. What if police keeps harassing the inhabitants of town A, fining them for every instance of littering or driving 1 mph faster than the speed limit, while they are pretty lenient in town B? Crime rates are going to be different even if the behaviour of people in the two towns is identical. Or what if a criminal mastermind liked the scenery in town A so much that s/he moved there, skewing the average completely?

But here's the catch: B's still shouldn't say that A's are criminals. Notice that in the case of medicine, we find the same individuals under the two bell-shaped curves. There, it makes sense to expect the change in the average to be reflected, to a certain extent, on the level of the individual. However, in the case of the two cities, there are different individuals under the two curves, and here, it is incorrect of the inhabitants of town B to ascribe a static quality to A's based on their average. A single average is a property of a group; it says nothing about an individual.

Accepting and propagating the idea that A's are worse and discriminating against them on that basis is not only incorrect and wrong, but will also genuinely and measurably harm the quality of life of A's. They will have lower incomes, will have worse education, work in worse conditions and have worse health because they are not given equal opportunities. This despite the fact that if you look at the graph, you'll find no one from A who committed more crimes than the person with the most convictions from B. Averages are not people.

But back to the painkillers. So, do you know what they call alternative medicine that's been proved to work?

Medicine.

(This is a quote from Tim Minchin's Storm. Take a look at it here or follow the links to listen to it on YouTube. It's good.)

Other types of "medicine" either couldn't move the average of wellbeing at all (save for the placebo effect), meaning that they have no measurable effect, or are downright harmful. Sure, Neighbour Nelly might've gotten better after that vinegar bath. But now you know.