Timeliness, Norms, and Mental Models, Oh My: Papers We’re Reading, Week 14

August 30, 2021  |  By: Allison White

According to a 2019 survey, about a third of American adults ages 25-73 (32%) didn’t plan to get a flu shot. This does not mean that the remaining ⅔ of Americans actually got their flu shot. In fact, in the 2019 – 2020 flu season, less than half (48.4%) of American ages 18+ got their flu shot. This is what we at Irrational Labs refer to as the intention-action gap.

Timeliness, Norms, and Mental Models, Oh My!

This paper presented results on a megastudy, a field experiment in which many interventions developed by different teams of scientists were tested in the same population on the same outcome. Twenty-six behavioral scientists generated 19 different text messaging interventions, with varied content and/or timing, all designed to boost adoption of the flu vaccine. Messages were sent to a total of 47,306 patients in fall 2020 who had new or routine (non-sick) primary care appointments scheduled at Penn Medicine or Geisinger Health. 

The team of researchers evaluated whether the participants received a flu shot on the date of their scheduled appointment, or in the three days leading up to it (when text messages started to be sent out). Some text messages included an opportunity to dedicate their vaccine to a loved one, or providing an anchor point for how their regions’ vaccination rates compared to others (i.e. “In YEAR, flu shots in YOUR REGION lagged behind the target rate of 70%.”). The text messaging nudges that performed best (4.6% boost in vaccination) were framed as reminders to get flu shots that were already reserved for the patient. Best performing interventions were also consistent with the sort of communications patients expected to receive from their healthcare provider—in other words, they were not surprising, casual, or interactive. 

You probably think I’m going to talk about how this study teaches us about all sorts of strategies to increase COVID-19 vaccination rates. However, these findings have much more wide-reaching lessons for all patient visits:

Make it Timely

Researchers tailored their intervention to encourage patients to get vaccinated who were already going in for a doctor’s appointment. When we want to get people to do something, the best way is to prompt them at the time when they’re most receptive. 

Leverage Norms

Framing the vaccine as already reserved for the patient makes it seem like the normal thing to do. People tend not to go against the grain unless they feel really strongly about something. 

Respect Mental Models

The best performing reminders aligned with patients’ pre-existing expectations about how Penn Medicine or Geisinger Health would communicate with them. When communicating with customers, it’s important to think about how their past experiences and perceptions might influence their interpretation. 


Milkman, et al. (2021). A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proceedings of the National Academy of Sciences of the United States of America 118 (20). May 18, 2021. Link: https://www.pnas.org/content/118/20/e2101165118#ref-2


Want to learn more about behavioral design? Join our Behavioral Economics Bootcamp today.


Our Services

From concept to code, explore how we get our hands dirty with research, product, and marketing challenges.

Our Areas of Expertise

Learn how we are helping change behaviors across the domains of health, education, finance, and more.

Join our Bootcamp

Understand your customers’ choices and learn how to change their behavior for the better — in our 8-week online Behavioral Design course.