…just saw that you worked for Professor Gigerenzer, so you know each other! Good stuff…

]]>Dear Hans,

Thank you very much for your great feedback and your recommending my blog to your students, I really appreciate that a lot!

Now to your point: my post was inspired by the great statistical series “Unstatistik des Monats” (something like “unstatistic of the month”): Der Impfstoff ist „zu 90 Prozent wirksam“ by my renowned colleague Professor Gerd Gigerenzer, former Director at the Max Planck Institute (unfortunately only available in German).

I think the approximation is good enough anyway because both groups were indeed about the same size. On top of that, the topic is quite complicated already, so I don’t want to complicate it any further.

Looking forward to your comments (or that of your students) on future posts

]]>thanks for a very interesting post! I generally like this blog a lot, and am recommending it to my students as a source of inspiration for data science projects. That said, a vaccination efficacy of 95% “doesn’t mean what you think it means” either, unfortunately.

Essentially, your above calculation of 1 – 8/162 computes vaccine efficacy as the complement to the simple post-test _odds ratio_ (`OR` = `N infected in vaccinated group` / `N infected in placebo group`). By contrast, the efficacy of vaccines is typically defined as the _relative risk reduction_ (`RRR`) attributed to the effect of the vaccination.

Although both these measures can be very similar, they are not identical (see Table 3 of https://doi.org/10.3389/fpsyg.2020.567817 for their definitions in terms of a 2×2 matrix). Their values happen to be identical when the vaccinated and placebo groups are exactly of the same size (i.e., `p_vacc = p_placebo = 1/2`). In most actual studies, however, the number of vaccinated people does not exactly match the number of individuals in the placebo group. Whereas your measure of `1 – OR` ignores group sizes, such imbalanced designs would yield different RRR values.

As most designs for evaluating vaccines strive for balance, the numeric deviations between both measures are typically small. However, as the title of your post implies to explain the meaning of “vaccine efficacy”, it seems worth pointing out that RRR would account for imbalanced designs, whereas the measure computed above only applies to a special case (when `p_vacc = 1/2`).

As the measure of `RRR` really _is_ very hard to understand, the icon arrays you show may help in rendering its consequences more transparent. (I somewhat doubt that seeing these consequences helps people to better understand the concept of `RRR` itself, but that would be yet another discussion.) A neat feature of using RRR is that its magnitude is independent of the overall population size `N`, which allows scaling down to more suitable subpopulations (as you do for creating the graphs).

I hope this clarifies things further — and I’m looking forward to future posts,

Hans

Great – thank you, Glen! I am the developer and maintainer of the OneR package, so I know how much work it is and, as I said, I don’t have the time at the moment.

In general, I am fine with nearly any form of cooperation as long as I am listed as one of the authors and a link to this post is provided.

What model of working together would you suggest/prefer?

]]>I’d be interested in helping turn this into a package and putting it on CRAN. ]]>