Case Example From the Nudge Study

Real World Evidence: Mobile Health (mHealth)

Section 4

Case Example From the Nudge Study


Christopher E. Knoepke, PhD, MSW

Jennifer D. Portz, PhD, MSW

Sheana Bull, PhD, MPH

Lisa Sandy, MPH

Thomas Glorioso, MS

Joy Wachtal, MPH

Phat Luong, MS

Adrian Hernandez, MD

Michael Ho, MD, PhD


Contributing Editor

Karen Staman, MS


Up to 50% of patients do not take their cardiovascular medications as prescribed, which results in increased morbidity, mortality, and healthcare costs (Brown and Bussell 2011). Interventions to improve adherence include patient education, reminders, pharmacist support, and financial incentives and have produced mixed results—some demonstrating benefits, but many producing small to negative results (NICE Clinical Guidelines No 76 2009). Adherence interventions have been limited by 1) including adherent patients who may not need an intervention; 2) resource-intensive approaches involving pharmacists and/or behavioral health; and 3) lack of attention to evidence-based strategies to motivate human behavior (Costa et al. 2015).

Brief behavioral interventions can influence decision-making and are impactful. Principles of behavioral economics have been incorporated into health interventions to “nudge” people to achieve improved health outcomes (Matjasko et al. 2016). A behavioral nudge is a small change in choice framing that alters people’s behavior in a predictable way. A prior study testing financial incentives through elimination of copayments for cardiovascular medications in the year after acute myocardial infarction improved adherence by 4% to 6% (Choudhry et al. 2014); however, financial incentives are not generalizable and are unlikely to be sustainable. Behavioral nudges, such as commitments (e.g., asking patients for demonstrated commitment to change through a pledge), norms (using examples of others who take action), and salience (making information or recommendations resonant through use of stories) build on a well-evidenced body of behavioral science theory and have been shown to improve health behaviors such as smoking cessation and weight loss (Matjasko et al. 2016). These have yet to be tested to improve medication adherence.

Mobile and digital technologies for health promotion and disease self-management offer an intriguing, low-resource, and as yet untested opportunity to adapt behavioral "nudges" using ubiquitous cell phone technology to facilitate medication adherence.

The objectives of the Personalized Patient Data and Behavioral Nudges to Improve Adherence to Chronic Cardiovascular Medications (Nudge) study, a two-part, multi-center study are as follows:

Objective 1: We developed and programmed a theoretically informed technology-based (a) nudge message library and (b) chat bot content library using multiple and iterative N of 1 within-subject studies to optimize content for a range of diverse patients. N of 1 participants came from three participating healthcare systems: University of Colorado Health System, VA Eastern Colorado Health Care System, and Denver Health Medical Center.

Objective 2: We conducted a pilot intervention to demonstrate feasibility of delivering the intervention and preliminary effects in two of the three healthcare systems. Throughout the process, we engaged patient, provider, and health systems stakeholders in designing, refining, and implementing the pilot intervention.

As the next step, we are building off of this work to conduct a pragmatic clinical trial to improve medication adherence and patient outcomes.


Objective 1

We drafted a complete library of proposed text messages, informed by principles of behavior change and behavioral economics, principally:

  1. Communicating social norms. Social norms can activate and guide behavior in positive ways when a message normalizes positive behaviors, such as medication adherence, placing non-adherence outside the definition of typical behavior. In other contexts, social norms have been shown to improve healthy food choices, physical activity, everyday health behaviors (e.g., using the stairs vs. elevators), and even reduce home energy use.
  2. Behavioral commitments. A behavioral commitment is affirmatively stating that the desired behavior (i.e., filling one’s prescription) will occur. Prior research has demonstrated a strong desire among individuals to act consistently with their prior commitments, and eliciting commitments to engage in a specific behavior has been shown to be effective at improving a range of behaviors, including substance use changes, safety-seeking behavior in the context of suicide prevention, and judicious use of antibiotics among clinicians. Commitments to fill one’s prescription are able to be elicited via text messaging and may lead to greater concordance between individuals’ commitment and their behaviors.
  3. Narrative stories: Narrative stories are increasingly recognized as an important way to increase vividness and comprehension of medical information and outcomes (Thompson and Kreuter 2014). One issue underlying medication non-adherence is likely a failure to recognize or understand the potential negative consequences the behavior, e.g., stroke, heart attack, or even death. Narrative interventions—particularly ones that describe stories of negative outcomes—may be particularly effective at helping patients concretely understand potential risks of non-adherence, spurring them to take action (improving medication adherence) to prevent negative outcomes.

Following the initial drafting of these messages, we conducted a series of N of 1 trials, with the a priori goal of refining the nudge messages, defining the best delivery method, and tailoring the interventions for diverse audiences, including Spanish-speaking patients and veterans. We engaged patients, providers, and health systems to provide feedback on the messages themselves, intervention design, and outcomes as well as engaging them in routine feedback during the study to help address potential barriers to implementation and help ensure the sustainability of the program. Concurrent with these activities, we established the IT infrastructure across the three healthcare systems for the study and built the library of nudge and chat bot messages.

Objective 2

We pilot tested the delivery and response to text messages, with specific interest in demonstrating feasibility of delivering the intervention and preliminary effects at two of the three healthcare systems, composed of: 1) generic medication refill reminder, 2) behavioral nudge, or 3) behavioral nudge plus artificially intelligent (AI) chat bot. We engaged patient, provider, and health systems leader stakeholders in designing, refining, and implementing the pilot intervention.

We beta-tested the delivery of the text messages and chat bot messages in a two-phased pilot randomized controlled trial within two healthcare systems to ensure feasibility and acceptability. Figure 1 offers a diagram of the process for message delivery for each of the study arms. We enrolled  210 patients from the two healthcare systems for the two-part pilot study, inclusive of patients who 1) received regular cardiovascular care through one of the two systems, 2) were prescribed at least one medication for long-term management of common cardiovascular conditions (Table 1), 3) did not return the opt-out consent form, and 4) had at least one medication refill at least 7 days overdue during the pilot period.

Figure 1. Pilot intervention

Patient Identification: Using administrative claims data, we developed codes to identify eligible patients currently being treated for one of the five cardiovascular conditions of interest, and in particular those who have been prescribed one or more of the classes of medications typically associated with the conditions in Table 1. Once an eligible patient had a “refill gap,” or a period after which they should have refilled medications but had not, of at least 7 days, we an sent opt-out consent letter. Once the deadline for response to the opt-out consent had expired, we randomized patients accordingly.

Cardiovascular conditions and associated classes of medications

Condition Classes of medications
Hypertension Beta-blockers (B-blocker), calcium channel blockers (CCB), angiotensin converting enzyme inhibitors (ACEi), angiotensin receptor blockers (ARB), thiazide diuretics
Hyperlipidemia HMG CoA reductase inhibitors (statins)
Diabetes Alpha-glucosidase inhibitors, biguanides, DPP-4 inhibitors, sodium glucose transport inhibitors, meglitinides, sulfonylureas, thiazolidinediones, and statins
Coronary artery disease PGY-2 inhibitors, B-blockers, ACEi or ARB, and statins
Atrial fibrillation Direct oral anticoagulants, B-blockers, CCB

Message Description: Patients randomized to receive text messages (Figure 1) received a combination of such messages so long as their index medications had not been refilled. As noted above, the content and framing of the messages themselves were informed by behavioral economics principles, and these messages will be compared to generic reminders (to account for the Hawthorne effect or simple forgetfulness on the part of the patient), and to a pre-programmed chatbot, which attempted to problem solve common barriers to medication adherence. Message types and timing of delivery include:

  1. Generic text: A generic reminder text was delivered to patients to refill their medication at days 1, 3, 5, and 7 after they been labeled as non-adherent. In the day 1 text message, patients had another opportunity to opt out of the study with text such as “text STOP if you wish to withdraw from this study.” The texts will stop once a patient has filled their medication.
  2. Behavioral nudge: A behavioral nudge text will be delivered to patients to remind them to refill their medications at days 1, 3, 5 and 7 after they have been labeled as non-adherent (Figure 2). In the day 1 text message, patients had another opportunity to opt out of the study with text such as “text STOP if you wish to withdraw from this study.” The texts will stop once a patient has filled their medication. The content of the behavioral nudge text messages varied with each text and was derived from the text message library built as part of Objective 1.

Figure 2. Schematic of the text messages for each of the text messaging

  1. Behavioral nudge plus AI chat bot: A behavioral nudge text was delivered to patients to remind them to refill their medications at days 1 and 3 after they had been identified as nonadherent. In the day 1 text message, patients had another opportunity to opt out of the study with text such as “text STOP if you wish to withdraw from this study.” The texts stopped once a patient had filled their medication. If the patient had not filled their medication on days 5 and 7, an AI would conduct interactive chat via a chat bot to assess barriers filling the medication as described in Objective 1.

The AI chat bot assessed for common barriers to medication adherence: 1) socioeconomic factors, 2) provider-patient/healthcare system factors; 3) condition-related factors; 4) therapy-related factors, and 5) patient-related factors using a script that we are currently employing in a medication adherence study. Communication about all of these barriers were pre-programmed in the chat bot automated program. For each barrier, the AI chat bot problem solves with the patient and identifies commonly used successful approaches to overcome barriers. It asks patients to choose and enact one solution to improve medication adherence.

For example, patients would be asked if they have difficulty remembering what medications to take and when to take them; those that do would be asked if using a medication diary, involving a caretaker, or setting an alarm on their phone would help. For those that agree to try a strategy, the AI chat program checks in one week later to see how the strategy is going. Those who do not agree or identify a strategy are offered other options, and the process repeated until they identify a strategy. If there are issues that arise that are not pre-programmed into the AI chat bot library, the AI chat bot refers the patient to the study pharmacist at each site for consultation and assistance with the issue. For example, a patient may have stopped taking his medication due to a side effect. The AI chat bot will document this information through interactive chat, then refer the patient to a study pharmacist to see if there are alternative medications. Dr. Bull, the co-PI of the study, has programmed libraries very similar to this AI chat bot approach and utilized them for behavior change in prior interventions.

Although behavioral nudges have not been tested for the improvement of medication adherence, the use of nudges builds on a substantial body of knowledge and has been shown to improve other health behaviors (Matjasko et al. 2016). If mobile and digital technologies can improve adherence to medication for patients with cardiovascular disease, there may be an opportunity to expand the use of nudges to facilitate medication adherence for a multitude of conditions.




back to top

Brown MT, Bussell JK. 2011. Medication adherence: WHO cares? Mayo Clin Proc. 86:304–314. doi:10.4065/mcp.2010.0575. PMID: 21389250.

Choudhry NK, Bykov K, Shrank WH, et al. 2014. Eliminating medication copayments reduces disparities in cardiovascular care. Health Aff (Millwood). 33:863–870. doi:10.1377/hlthaff.2013.0654. PMID: 24799585.

Costa E, Giardini A, Savin M, et al. 2015. Interventional tools to improve medication adherence: review of literature. Patient Prefer Adherence. 9:1303–1314. doi:10.2147/PPA.S87551. PMID: 26396502.

Matjasko JL, Cawley JH, Baker-Goering MM, Yokum DV. 2016. Applying behavioral economics to public health policy. Am J Prev Med. 50:S13–S19. doi:10.1016/j.amepre.2016.02.007. PMID: 27102853.


NICE Clinical Guidelines No 76. 2009. Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence.  Chapter 8. Interventions to Increase Adherence to Prescribed Medicine.

Thompson T, Kreuter MW. 2014. Using written narratives in public health practice: a creative writing perspective. Prev Chronic Dis. 11:130402. doi:10.5888/pcd11.130402. PMID: 24901794.

Version History

Published March 16, 2020


Knoepke CE, Portz JD, Bull S, et al. Real World Evidence: Mobile Health (mHealth): Case Example From the Nudge Study. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials. Bethesda, MD: NIH Health Care Systems Research Collaboratory. Available at: Updated March 26, 2020. DOI: 10.28929/124.