- Journal List
- Elsevier - PMC COVID-19 Collection
- PMC9870038
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
Vaccine. 2023 Feb 10; 41(7): 1310–1318.
Published online 2023 Jan 23. doi:10.1016/j.vaccine.2022.12.031
PMCID: PMC9870038
PMID: 36697313
Tanya R. Myers,a,⁎ Paige L. Marquez,a Julianne M. Gee,a Anne M. Hause,a Lakshmi Panagiotakopoulos,a Bicheng Zhang,a Isaac McCullum,a Charles Licata,a Christine K. Olson,a Shakeeb Rahman,b Sharon B. Kennedy,b Matthew Cardozo,b Chirayu R. Patel,b Liza Maxwell,b Joel R. Kallman,b David K. Shay,a and Tom T. Shimabukuroa
Author information Article notes Copyright and License information PMC Disclaimer
Associated Data
- Supplementary Materials
- Data Availability Statement
Abstract
The Centers for Disease Control and Prevention (CDC) developed and implemented the v-safe after vaccination health checker (v-safe) to monitor COVID-19 vaccine safety and as an active surveillance supplement to existing CDC vaccine safety monitoring programs. V-safe allows persons who received COVID-19 vaccines to report on post-vaccination experiences and how symptoms affected their health at daily, weekly, and monthly timepoints after vaccination. Text message reminders are sent linking to Internet-based health check-in surveys. Surveys include questions to identify v-safe participants who may be eligible to enroll in a separate pregnancy registry activity that evaluates maternal and infant outcomes in those pregnant at the time of vaccination or receiving vaccine in the periconception period.
We describe the development of and enhancements to v-safe, data management, promotion and communication to vaccination sites and partners, publications, strengths and limitations, and implications for future systems. We also describe enrollment in v-safe over time and demographics of persons participating in v-safe during the first year of operation (December 14, 2020 – December 13, 2021). During this time, 9,342,582 persons submitted 131,543,087 v-safe surveys. The majority of participants were female (62.3%) and non-Hispanic White (61.2%); median age was 49.0years. Most participants reported receiving an mRNA COVID-19 vaccine as their first recorded dose (95.0%).
V-safe contributed to CDC’s vaccine safety assessments for FDA-authorized COVID-19 vaccines by enabling near real-time reporting of reactogenicity once the COVID-19 vaccination program began in the community, encouraging reports to the Vaccine Adverse Event Reporting System and facilitating enrollment in a large post-vaccination pregnancy registry. Given that v-safe is an integral component of the most comprehensive safety monitoring program in U.S. history, we believe that this approach has promise as a potential application for future pandemic response activities as well as rollout of novel vaccines in a non-pandemic context.
Keywords: COVID-19 vaccine, V-safe, Active surveillance, Vaccination, Vaccine adverse event, Vaccine safety
1. Introduction
Vaccine safety monitoring after licensure or authorization of vaccines is a critical function to ensure that timely and transparent information related to the safety of vaccines can be communicated to public health officials, healthcare providers, and the public. Such monitoring is particularly important to identify rare adverse events that may not have been identified in clinical trials and to monitor vaccine safety in populations that were not included in initial trials, e.g., pregnant women. Beginning in mid-2020, the Centers for Disease Control and Prevention (CDC) and the U.S. Food and Drug Administration (FDA) began comprehensive planning and preparation to implement safety monitoring for a large-scale national COVID-19 vaccination program, anticipated to start by the end of 2020. Because of the expedited vaccine development and testing process and the urgency to begin vaccination to protect individuals against severe illness and death related to COVID-19 infection, COVID-19 vaccines were initially administered entirely under Emergency Use Authorization (EUA) [1]. Both CDC and FDA were committed to implementing a robust safety monitoring program to identify and assess potential safety issues as quickly as possible.
Plans for safety monitoring included existing systems such as the Vaccine Adverse Event Reporting System (VAERS), co-managed by CDC and FDA [2], and large-linked electronic health record and claims-based systems such as the Vaccine Safety Datalink (VSD), the Biologics Effectiveness and Safety System (BEST) and the Center for Medicare and Medicaid Services (CMS) [3], [4]. VAERS is a national spontaneous reporting (passive surveillance) system that serves to identify potential vaccine safety signals for further evaluation. VAERS generates timely data but is limited by incomplete reporting of events to VAERS, and varying quality and completeness of reported information. Healthcare providers who administer COVID-19 vaccines are required to report specific adverse events to VAERS as part of the COVID-19 vaccine provider agreement; more information about these reporting requirements is available at vaers.hhs.gov/reportevent.html.
VSD, BEST, and CMS are utilized for active surveillance. These systems also support investigation of vaccine safety signals via methods that include comparisons of possible adverse events in vaccinated and unvaccinated persons, current versus historical cohorts, or in exposed and unexposed periods of time (near and far from vaccine event) for vaccinated individuals. However, although these systems rely on electronic health records, they may also require manual medical record review and/or linkage with insurance claims data leading to variable lag times until pertinent data for conducting safety assessments become available.
CDC and FDA recognized a need to supplement passive VAERS surveillance during the early stages of the COVID-19 vaccination program when limited vaccine doses were available from the other vaccine safety surveillance systems. Both agencies were also interested in rapidly collecting information on COVID-19 vaccination during pregnancy. To address these data needs, the CDC, in collaboration with technology company Oracle (https://www.oracle.com), developed and implemented v-safe, a new vaccine safety monitoring system specifically designed to conduct near real-time active surveillance for COVID-19 vaccine safety monitoring. In preparation for the start of vaccine distribution in the United States, v-safe opened for registration on December 13, 2020 and began collecting data on December 14, 2020.
2. v-safe: A new smartphone-based national surveillance system for vaccine safety
The v-safe after vaccination health checker is a voluntary, self-enrollment program that capitalizes on smartphone technology to conduct active surveillance of post-vaccination symptoms and diagnoses of new health conditions. Once enrolled, Short Message Service (SMS, also referred to as text messaging), invitations ask enrollees to complete simple, web-based surveys at specific timepoints after vaccination. These text invitations include a uniform resource locator (URL) that links directly to the survey within the participant’s v-safe account. Surveys received in the first week after vaccination seek to identify specific symptoms that may indicate a local or systemic reaction as well as any other symptoms or health conditions, while surveys at later timepoints (weeks or months after vaccination) ask about any new or worsening symptoms or health conditions. Identification of potentially eligible enrollees in CDC’s COVID-19 vaccine pregnancy registry was exclusive to v-safe participants, using survey questions designed to identify those who were pregnant at the time of vaccination or became pregnant after vaccination (5). User interactions and data flow are shown in Fig. 1. Full health survey content is included in the supplemental material (Supplement 1).
Fig. 1
V-safe after vaccination health checker system data user interaction and data flow*, *Blue boxes indicate v-safe administrative access activities such as initiating data download; red boxes indicate v-safe participant activities. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
To participate in v-safe, participants must self-register in v-safe and enter information about their vaccines to begin receiving health surveys. The starting point for registration is the v-safe landing page, vsafe.cdc.gov, which provides a brief description of v-safe and directs participants to either an online registration page or an online access page for returning to existing v-safe accounts. To register, participants are required to agree to accept text messages and enter first and last name, date of birth, sex, zip code, and mobile phone number. Once these items have been submitted to v-safe, the participant is asked to verify their registration activity by entering a six-digit verification code received by text message. After verification, participants are asked to enter all COVID-19 vaccines received, starting from their first dose, including manufacturer name and date of receipt. Although v-safe participants are encouraged to enroll immediately after their first dose of COVID-19 vaccine, enrollment can occur at any time after vaccination and after any dose.
Self-registration and enrollment of dependents is limited to those 16years and older. The dependent feature allows parents and guardians to register, record vaccines and complete surveys for children and adolescents who are younger than 16years. Because dependents of any age can be added, the dependent feature can also be used to enroll family members or friends who don’t have smartphone access or prefer not to enroll themselves, such as elderly parents or neighbors.
Upon entry of vaccine information, participants receive text reminders with URL links to secure internet-based health surveys, with the appropriate survey timing determined based on the date of last COVID-19 dose recorded in v-safe. Participants are asked to complete daily surveys on days 0–7 following receipt of each dose of COVID-19 vaccine (day 0 is the day of vaccination), then weekly through day 42 (6weeks) after vaccination, and again at 3, 6 and 12months. Surveys are available for completion starting at 2pm local time (based on the zip code entered by the participant at registration), with daily surveys remaining available until midnight local time that day. Surveys at later timepoints remain open until midnight on the 6th day after survey receipt. Participants who enter their vaccine dose into v-safe after day 0 or who miss a health survey cannot access surveys for timepoints that have already passed. Once sufficient calendar time has elapsed for a participant to be eligible for another dose (either to complete a primary series of COVID-19 vaccine with a second dose or to receive a booster dose), v-safe sends a text reminder that includes a URL link to access their account. Participants can also visit the v-safe landing page at any time to access their account and log another dose. The total number of surveys that participants are eligible to receive for a particular dose depends on whether doses are recorded in v-safe on the same day as received and whether additional doses are recorded in v-safe at a later time.
Daily survey content includes questions about local reactions at the injection site (pain, redness, swelling, itching) and systemic reactions (chills, headache, joint pain, muscle or body aches, fatigue or tiredness, nausea, vomiting, diarrhea, abdominal pain, non-injection site rash, and fever). Participants who report fever are also asked to record their highest temperature for that day (if known); for local reactions and solicited reactions (except fever), they are asked to characterize severity for each reported symptom. For adults and verbal children, severity is defined as mild when symptoms are noticeable but aren’t a problem, as moderate when symptoms limit normal daily activities, and as severe when symptoms make normal daily activities difficult or impossible. Specific definitions of mild, moderate, and severe symptoms for non-verbal children vary by age and are described in the full survey content in Supplement 1. Participants can also record any other symptoms or health conditions they would like to report to v-safe by typing in a free text field. Weekly surveys at days 14, 21, 28, 35 and 42 and surveys at 3, 6 and 12months after vaccination are simplified and do not include questions about solicited reactions; these surveys ask participants to report any worsening symptoms or new health conditions since the previous survey in a free text field. By design, the survey structure includes questions on daily and weekly surveys to capture other symptoms or health conditions that are acute (haven’t been previously reported); while duration of the solicited symptoms can be assessed by frequency of reporting over the surveys completed on days 0–7, the survey questions are not structured to assess duration of symptoms and health conditions reported as free text.
All surveys ask participants to indicate whether any of the symptoms or health conditions they reported caused a health impact (defined as being unable to work or attend school, being unable to do normal daily activities, or seeking care from a doctor or other healthcare professional). Any combination of these health impacts can be selected. Participants who indicate they sought medical care are asked to identify the source of care received (virtual, outpatient or urgent care, emergency department, or hospitalization) and receive telephone outreach from a live agent at a v-safe call center to encourage them to complete a report to VAERS; the call center agents offer to complete the report during the phone call. In cases where the participant does not wish to complete the report during the phone call but indicates willingness to report on their own, the agents provide information so that the participant can complete the report at a later time. No clinical guidance is provided during the follow-up call; participants who have specific questions about how a symptom should be treated are advised to seek guidance from their doctor or healthcare provider.
All participants who did not indicate sex as male during registration are also asked questions on select surveys to identify potential pregnancies and facilitate enrollment into the COVID-19 vaccine pregnancy registry [5]. The first survey completed for each dose asks whether the participant was pregnant when they received that dose. If the participant answers “no” or “I don’t know” to this question, a follow-up question at select daily and weekly timepoints (day 21, day 42, 3month, 6month, 12month) for that dose asks if they have had a home or laboratory pregnancy test that was positive since their last COVID-19 vaccination. Telephone outreach from the CDC’s COVID-19 vaccine pregnancy registry team (a separate activity from the v-safe call center mentioned above) identifies those who may be eligible and consent to participate in the pregnancy registry, a research activity that conducts follow-up of maternal, birth, and infant outcomes after vaccination during pregnancy or the periconception period of 30days prior to the last menstrual cycle through 14days afterward [5].
All v-safe resources (including text messaging and surveys) are available in English, Chinese, Korean, Spanish and Vietnamese. The CDC website landing page for v-safe (vsafe.cdc.gov) provides a link to online resources to learn more about v-safe or the COVID-19 vaccine pregnancy registry, including instructions on how to enroll, add dependents, complete a health survey, and access v-safe in a different language. Participants who need further technical assistance beyond online resources are directed to contact v-safe support through CDC-INFO (CDC’s national contact center) which links to live agents via telephone or allows online submission of questions. For participants who wish to stop receiving text message reminders, standard opt-out language allows them to respond “STOP” to any v-safe text message. Participants can also restart text message reminders by responding “RESTART” to any v-safe text message.
3. Development and promotion of v-safe
The Oracle Health Management System is a multi-functional COVID-19 response system built and donated to the U.S. Department of Health and Human Services to enable COVID-19 testing and monitor patients who participated in clinical trials for COVID-19 therapeutics after discharge from hospital. Upon recognizing that this system provided capabilities and technologies that could address CDC’s desire to supplement existing vaccine safety systems with an active surveillance component, CDC and Oracle teams worked together to quickly develop the v-safe system. Oracle used Application Express (APEX), a low-code application development platform within its cloud infrastructure to design and deploy v-safe. User feedback from testing during development was used to simplify the registration process and the user experience overall. Ease of use for self-registration and vaccine entry was a priority during v-safe development to improve the likelihood that participants would successfully enroll. Development of the survey questions prioritized keeping each survey brief with simple questions because users are asked to participate over multiple survey timepoints extending to a year after the last dose of COVID-19 vaccine that is recorded in v-safe. The platform was designed to restrict access to persons physically located in states or territories of the United States.
Because v-safe requires self-registration, CDC provided all vaccine partners with information sheets in electronic format for provision to vaccine recipients. These partners included jurisdictional clinics and vaccination sites, retail pharmacies, long-term care facilities, Federal Emergency Management Agency and Health Resources and Services Administration, and other federal entity partners. The information sheets included a description of v-safe, a QR (quick response) code for scanning to access the landing page online, and the URL for entry into a web browser if preferred over scanning a QR code. The information sheet also contained contact information (telephone and e-mail address) to reach CDC-INFO for questions or technical assistance. Vaccination sites were not mandated to provide the information sheets to vaccine recipients, but CDC encouraged partners to distribute the information sheets to each vaccination site for provision in printed format; some vaccination sites were able to incorporate information about v-safe into electronic messaging (e-mail or text-based) sent to individuals either shortly before or after vaccination. CDC also promoted the new system on its online COVID-19 vaccine resource pages and through social media channels.
4. Post-launch enhancements to v-safe
Since the launch of v-safe on December 13, 2020, enhancements have been made to meet the needs of v-safe participants, capture additional data elements, and adapt to changes in the vaccination program. Multi-lingual functionality was added to the v-safe platform in March 2021, allowing users to register and view surveys in Chinese, Korean, Spanish, and Vietnamese in addition to English. The original structure of the platform allowed for self-registration and participation in v-safe for persons ages 16years and older, accommodating the lowest age eligible for vaccination under the initial EUA for the Pfizer-BioNTech COVID-19 vaccine. When the EUA was expanded to allow for vaccination in persons as young as 12years, a dependent feature was added to v-safe so that parents can register their dependents and complete health surveys on their behalf. Upon authorization of vaccine for younger children, modified surveys were enabled for dependents younger than 3years so that parents and guardians can report on the health experiences of non-verbal children.
While the original v-safe registration and vaccine entry processes were designed to require as little personal information as possible and to reflect the recommended series of available vaccines in the United States, data needs and changes to recommendations have necessitated modifications to include additional data elements. To more completely describe the demographics of v-safe participants, questions on race and ethnicity were added in February 2021 as a component of the initial health survey completed by participants. When recommendations were developed for coadministration of other vaccines at the same time as COVID-19 vaccines, v-safe added a question to allow identification of the type of vaccines received on the same day as COVID-19 vaccines. Finally, vaccine entry processes were adapted to allow for recording and completing surveys beyond the primary series, initially for a single booster dose and then expanding as needed for additional booster dose recommendations.
5. Data management and analysis activities
Data collected during registration and health surveys are stored in a secure server. Download of the data to CDC consists of multiple file sets, including registration, vaccination, race and ethnicity, and health survey files. Each entry in these datasets includes a unique v-safe registrant code that can be used to link data elements for individual v-safe participants. A CDC v-safe data manager downloads files to a secure server each workday for newly accrued data, and on a weekly basis, cumulative data files without variables containing Personal Identifiable Information (PII) are created for use by data analysts. This data flow between participants, v-safe and CDC is depicted in Fig. 1. The analyses described here were conducted in SAS software (version 9.4, SAS Institute). Data on first doses COVID-19 vaccine administered in the study period were sourced from CDC’s COVID-19 Data Tracker.
The v-safe platform supports active surveillance for public health purposes. Registration in v-safe is voluntary, and registered participants opt-in to receive health surveys. This surveillance activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC policy (45C.F.R. part 46, 21C.F.R. part 56; 42 U.S.C. §241(d); 5 U.S.C. §552a; 44 U.S.C. §3501 et seq.). Participants in v-safe are informed that their information is protected by administrative, technical, and physical measures that safeguard the confidentiality, integrity, and privacy of personal information, and that to the extent v-safe uses existing information systems managed by CDC, FDA, and other federal agencies, the systems employ strict security measures appropriate for the level of sensitivity of the data.
6. Early participation in v-safe
From the start of the U.S. COVID-19 vaccination program on December 14, 2020 through December 13, 2021, 9,806,588 people registered. Of these, 9,342,582 (95.3%) completed at least one health survey and were considered active v-safe participants; during this timeframe, participants completed a total of 131,543,087 health surveys. Demographic characteristics of active participants by vaccines received are shown in Table 1. As compared to the U.S. population who received at least one dose of COVID-19 vaccine during this time period (data from COVID-19 Data Tracker), a higher proportion of v-safe participants identified as female (62.3%) and non-Hispanic White (61.2%); the median age of participants was 49.0years, with lower proportions of children and adolescents and adults ages 75 and older than the U.S. population receiving at least one dose of COVID-19 vaccine.
Table 1
Demographics and activity by dose for active participantsa in v-safe, December 14, 2020 – December 13, 2021.
| Characteristic | Pfizer-BioNTechb N=4694527 | Modernab N=4184910 | Janssenb N=462071 | Unknownb N=1074 | Total N=9342582 | COVID-19 vaccinated U.S. population in study periodc | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| n | % | n | % | n | % | n | % | n | % | % | |
| Sex | |||||||||||
| Female | 2,915,900 | 62.11 | 2,639,234 | 63.07 | 267,390 | 57.87 | 543 | 50.56 | 5,823,067 | 62.33 | 51.84 |
| Male | 1,735,902 | 36.98 | 1,503,489 | 35.93 | 192,444 | 41.65 | 515 | 47.95 | 3,432,350 | 36.74 | 47.28 |
| Other | 5,927 | 0.13 | 4,432 | 0.11 | 780 | 0.17 | 3 | 0.28 | 11,142 | 0.12 | 0.87 |
| Prefer not to say | 36,798 | 0.78 | 37,755 | 0.90 | 1,457 | 0.32 | 13 | 1.21 | 76,023 | 0.81 | n/a |
| Age Group (years)d | |||||||||||
| 0–4 | 3e | 0.00 | 0 | --- | 0 | --- | 0 | --- | 3 | 0.00 | 0.01 |
| 5–11 | 51,621 | 1.10 | 24e | 0.00 | 11e | 0.00 | 61 | 5.68 | 51,717 | 0.55 | 2.53 |
| 12–17 | 187,101 | 3.99 | 1,435e | 0.03 | 428e | 0.09 | 116 | 10.80 | 189,080 | 2.02 | 6.67 |
| 18–49 | 2,307,945 | 49.16 | 1,908,767 | 45.61 | 265,887 | 57.54 | 593 | 55.21 | 4,483,192 | 47.99 | 44.15 |
| 50–64 | 1,265,662 | 26.96 | 1,185,640 | 28.33 | 156,877 | 33.95 | 175 | 16.29 | 2,608,354 | 27.92 | 23.65 |
| 65–74 | 655,223 | 13.96 | 805,837 | 19.26 | 32,753 | 7.09 | 96 | 8.94 | 1,493,909 | 15.99 | 13.68 |
| ≥75 | 226,972 | 4.83 | 283,207 | 6.77 | 6,115 | 1.32 | 33 | 3.07 | 516,327 | 5.53 | 9.31 |
| Race and Ethnicity | |||||||||||
| Hispanic or Latino | 535,530 | 11.41 | 431,254 | 10.30 | 59,199 | 12.81 | 193 | 17.97 | 1,026,176 | 10.98 | 15.02 |
| Non-Hispanic or Latino | |||||||||||
| American Indian or Alaska Native | 15,742 | 0.34 | 17618 | 0.42 | 1576 | 0.34 | 9 | 0.84 | 34945 | 0.37 | 0.68 |
| Asian | 360,630 | 7.68 | 243675 | 5.82 | 31999 | 6.93 | 130 | 12.10 | 636434 | 6.81 | 4.94 |
| Black | 298,579 | 6.36 | 239407 | 5.72 | 31228 | 6.76 | 159 | 14.80 | 569373 | 6.09 | 7.48 |
| Native Hawaiian or Pacific Islander | 13,285 | 0.28 | 10910 | 0.26 | 1325 | 0.29 | 8 | 0.74 | 25528 | 0.27 | 0.23 |
| White | 2,817,982 | 60.03 | 2602554 | 62.19 | 298887 | 64.68 | 333 | 31.01 | 5719756 | 61.22 | 42.37 |
| Multi-racial | 79,777 | 1.70 | 56831 | 1.36 | 8237 | 1.78 | 21 | 1.96 | 144866 | 1.55 | 1.68 |
| Other | 30,673 | 0.65 | 22715 | 0.54 | 3140 | 0.68 | 12 | 1.12 | 56540 | 0.61 | 2.98 |
| Race unknown or not reported | 16,005 | 0.34 | 13489 | 0.32 | 1567 | 0.34 | 5 | 0.47 | 31066 | 0.33 | n/a |
| Unknown ethnicity | |||||||||||
| American Indian or Alaska Native | 1,808 | 0.04 | 2162 | 0.05 | 188 | 0.04 | 5 | 0.47 | 4163 | 0.04 | n/a |
| Asian | 27,013 | 0.58 | 20153 | 0.48 | 2967 | 0.58 | 21 | 1.96 | 49884 | 0.53 | n/a |
| Black | 17,642 | 0.38 | 16334 | 0.39 | 2151 | 0.47 | 39 | 3.63 | 36166 | 0.39 | n/a |
| Native Hawaiian or Pacific Islander | 1,399 | 0.03 | 1183 | 0.03 | 155 | 0.03 | 3 | 0.28 | 2740 | 0.03 | n/a |
| White | 70,103 | 1.49 | 71646 | 1.71 | 7824 | 1.69 | 34 | 3.17 | 149607 | 1.60 | n/a |
| Multi-racial | 4,901 | 0.10 | 3514 | 0.08 | 538 | 0.12 | 7 | 0.65 | 8960 | 0.10 | n/a |
| Other | 9,589 | 0.20 | 7983 | 0.19 | 1151 | 0.25 | 11 | 1.02 | 18734 | 0.20 | n/a |
| Race unknown or not reported | 88,329 | 1.88 | 77569 | 1.85 | 10055 | 2.18 | 83 | 7.73 | 176036 | 1.88 | n/a |
| Missing | 305,540 | 6.51 | 345913 | 8.27 | 154 | 0.03 | 1 | 0.09 | 651608 | 6.97 | 24.61 |
| Participants completing at least one survey by Dec. 13, 2021 | |||||||||||
| Dose 1 | 3,975,829 | 84.7 | 3,697,178 | 88.3 | 452,483 | 97.9 | 522 | 48.6 | 8,126,012 | 87.0 | n/a |
| Dose 2 | 3,545,310 | 75.5 | 3,134,156 | 74.9 | 59,909 | 13.0 | 323 | 30.1 | 6,739,698 | 72.1 | n/a |
| Dose 3 | 590,323 | 12.6 | 517,312 | 12.4 | 780 | 0.2 | 288 | 26.8 | 1,108,703 | 11.9 | n/a |
Open in a separate window
aActive participants completed at least one health survey in v-safe, after one or more doses of COVID-19 vaccines.
bManufacturer of first dose of vaccine reported to v-safe by participant; some participants reported a different vaccine manufacturer for subsequent doses.
cIncludes persons receiving at least one dose of COVID-19 vaccine in U.S. states and territories from December 14, 2020 – December 13, 2021; source: Centers for Disease Control and Prevention. COVID Data Tracker.
dAge at first recorded dose in v-safe.
eFirst recorded dose manufacturer, unknown whether entries were in error or represent unauthorized use of vaccines.
Monthly v-safe registration numbers for active participants are shown in Fig. 2, as well as the percent of COVID-19 vaccinated persons in the U.S. participating in v-safe after receiving at least one dose of vaccine. Participation in v-safe was highest in the early months of the COVID-19 vaccine rollout, with more than 10% of vaccine recipients participating in December 2020 and January 2021; by December 13, 2021, active v-safe participants represented 2.9% of the U.S. population that had received COVID-19 vaccines. A majority of participants in the study cohort (n=7,492,700; 80.2%) registered and completed their (or their dependent’s) first survey within 7days after receiving a first dose of COVID-19 vaccine. These participants could complete up to 8 surveys in days 0–7 after vaccination if they registered their dose on the day of vaccination; the median (IQR) number of daily surveys completed in the first week after vaccination was 6 (4–7) after dose 1 and 7 (4–8) after dose 2. Participants could complete up to 5 weekly surveys at weeks 2–6; the median (IQR) number of weekly surveys completed was 2 (1–2) after dose 1 of mRNA COVID-19 vaccine and 5 (4–5) after dose 2 of mRNA COVID-19 vaccine. For the single-dose Janssen vaccine, the median (IQR) number of weekly surveys completed was 5 (4–5) after dose 1. Fewer weekly surveys were completed for dose 1 of mRNA vaccines due to entry of the second dose in the primary series and subsequent reset of survey timing.
Fig. 2
V-safe registrations by month, December 14, 2020 – December 13, 2021. Registrants included if they completed at least one health survey at any time during December 14, 2020 – December 13, 2021; persons with≥1 COVID-19 vaccine dose in U.S. states and territories sourced from Centers for Disease Control and Prevention, COVID Data Tracker.
To assess survey completion over time by the study cohort, including throughout the first week, we identified registrants who submitted surveys within a day of vaccination (defined as completion of a day 0 or day 1 survey) and calculated the proportion of daily and weekly surveys submitted by this cohort at each survey timepoint by manufacturer and dose (Table 2). For dose 1 of mRNA vaccine, survey completion dropped considerably at week 3 (day 21) and later timepoints as participants entered their second dose of vaccine and initiated dose 2 surveys. For all other doses, survey completion was slightly lower at later timepoints but with a majority still participating through the weekly timepoints. At the conclusion of the project, a more complete assessment of survey attrition over time, including monthly timepoints and accounting for progression to next dose is planned.
Table 2
Survey completion over timepoints, in active participants* who initiated surveys on day 0 or day 1 for each dose.
| mRNA vaccines (Pfizer-BioNTech and Moderna) | Janssen vaccine | |||||
|---|---|---|---|---|---|---|
| Survey Timepoint | Dose 1 N=5902402 | Dose 2 N=4516506 | Dose 3 N=662316 | Dose 1 N=348783 | Dose 2 N=7062 | Dose 3 N=705 |
| % Completion | % Completion | % Completion | % Completion | % Completion | % Completion | |
| Day 0 | 77.7 | 79.8 | 61.1 | 74.1 | 67.3 | 61.1 |
| Day 1 | 87.9 | 92.5 | 94.4 | 90.9 | 89.8 | 91.2 |
| Day 2 | 79.9 | 85.6 | 86.6 | 82.2 | 81.4 | 81.6 |
| Day 3 | 76.8 | 82.2 | 84.8 | 76.6 | 78.2 | 80.0 |
| Day 4 | 73.2 | 79.1 | 81.9 | 71.2 | 74.5 | 75.3 |
| Day 5 | 69.2 | 76.6 | 79.2 | 66.5 | 71.1 | 74.2 |
| Day 6 | 66.3 | 74.6 | 77.5 | 61.6 | 69.2 | 69.5 |
| Day 7 | 77.7 | 83.4 | 86.5 | 69.2 | 80.0 | 79.7 |
| Week 2 (day 14) | 77.2 | 81.8 | 85.3 | 67.1 | 78.3 | 79.1 |
| Week 3 (day 21) | 44.9 | 78.2 | 83.5 | 63.2 | 75.6 | 75.9 |
| Week 4 (day 28) | 9.8 | 75.5 | 81.9 | 60.0 | 74.4 | 74.8 |
| Week 5 (day 35) | 1.3 | 73.0 | 80.7 | 57.4 | 73.3 | 72.8 |
| Week 6 (day 42) | 0.4 | 74.0 | 81.0 | 57.7 | 73.3 | 73.2 |
Open in a separate window
*Active participants are those who completed at least one health survey in v-safe December 14, 2020 – December 13, 2021; survey completion metrics include surveys completed through November 6, 2022.
For 82,944,094 surveys completed March 7, 2021 – December 13, 2021 (the study timeframe during which v-safe was available in multiple languages), English was the most frequently utilized language by participants (99.50% of all surveys), followed by Spanish (0.34%) and Chinese (0.13%). Irrespective of language selected for the health check-in survey, the median time for completion of surveys was less than one minute. The lengthier daily surveys had a median completion time (IQR) of 0.45 (0.27–0.80) minutes, while the shorter weekly and monthly surveys had a median completion time (IQR) of 0.32 (0.20–0.52) minutes.
As of December 13, 2021, over 21,000 VAERS reports had been completed during outreach from the v-safe call center; reports to VAERS from v-safe participants are not summarized further here, as they are incorporated into analyses of all reports received to VAERS after COVID-19 vaccine.
7. v-safe publications
Data from v-safe have been included in multiple publications describing both early safety monitoring of all COVID-19 vaccines after initial authorization in the United States as well as findings for Pfizer-BioNTech and Moderna vaccines after six months of use (5–9). Additional publications have described reporting to v-safe after vaccinations in children and adolescents and after additional or booster doses received after completion of the primary series of COVID-19 vaccines (10–14). As additional COVID-19 vaccines are authorized and as current authorizations are amended to allow for vaccination in younger age groups, future publications will include v-safe data analyses for novel vaccines or new age groups. A current list of CDC’s vaccine safety publications is available online and is updated as new manuscripts are published (https://www.cdc.gov/vaccinesafety/research/publications/index.html).
8. Strengths and limitations of v-safe
An important strength of the v-safe platform is the ability to engage directly with vaccine recipients throughout the United States and territories and gather near real-time data about health experiences after vaccination. The use of SMS messaging provided an efficient way to remind participants to complete surveys as well as to seek primary series or booster doses and record these doses in v-safe. The number of people reporting to v-safe has far exceeded the number of participants in clinical trials of COVID-19 vaccines, providing a large population size to support descriptions of reported events after vaccination. Anecdotally, some v-safe participants have reported through social media posts that the individualized follow-up through periodic health check-ins have made a positive impression regarding CDC’s vaccine safety monitoring [15], [16], [17], [18], [19], [20]. Another strength of v-safe is that health surveys are not initiated because a post-vaccination symptom or health event is present; instead, participants can report how they are feeling, including survey responses that describe the absence of new symptoms or health events. To encourage continued participation, the surveys were designed to be easy to navigate and brief, and in practice, the average time to complete a survey was less than a minute. Finally, the platform enabled direct outreach to potential participants for additional targeted vaccine safety evaluation activities, such as a study of the association between prior confirmed SARS-COV-2 infection and severe systemic adverse events after mRNA COVID-19 vaccination [21] and the COVID-19 vaccine pregnancy registry [5], [22].
An important limitation of v-safe is that, like any voluntary reporting system, participants are not a random sample of all vaccinated persons in the United States. Findings drawn from the self-selected population of participants in v-safe may not be generalizable to the full U.S. vaccinated population. The earliest registrants in v-safe were predominantly younger, likely healthy essential workers (healthcare staff and first responders), but also included a smaller population of long-term care facility residents enrolled; this reflects the population recommended to receive vaccine in the earliest days of the vaccination program rollout. Although v-safe does not collect data on occupation, healthcare workers may have been more likely to register and participate actively in v-safe due to the early availability of COVID-19 vaccines for this group.
Another limitation of the v-safe platform is that reporting is incomplete. Although invited to all surveys for which they were eligible based on time since vaccination, participants may have been more likely to complete surveys if experiencing symptoms post-vaccination. Because the platform relies on delivery of text messages to mobile phones, some surveys may have been missed due to mobile outages or due to other technical malfunctions in the delivery process. Although maintenance windows for the platform were scheduled for low activity times whenever possible, occasional system outages may have also prevented survey completion. Finally, although many participants continued completing surveys at longer timepoints, attrition over time also contributes to incomplete reporting.
The selection of a smartphone-based platform also limits participation to people who use these devices. A recent estimate of smartphone usage in the United States found that 85% of adults reported having a smartphone, and that smartphone usage varied most substantially by age, education, and income level [23]. We relied heavily in the early stages of the COVID-19 vaccination program on voluntary promotion of v-safe at vaccination sites, which probably led to under-representation from sites that did not provide information about how to register and participate in v-safe. Another limitation is that the health surveys do not ask about resolution of previously reported symptoms or health events; therefore, while we can assess the duration of solicited reactogenicity in surveys completed in the first week, we do not have data regarding resolution of previously reported unsolicited symptoms to aid in identifying acute versus chronic complaints. The health surveys also do not ask questions about underlying medical conditions, and some reports of new or worsening symptoms may be related to health conditions diagnosed prior to COVID-19 vaccination. Therefore, while interpretation of data regarding reports of solicited reactions to v-safe is relatively straightforward, reports of other symptoms or health events to v-safe are more difficult to interpret from v-safe data alone. Additional information on these symptoms and health events is, however, captured in VAERS if medical attention was sought and if the participant can be contacted and is willing to complete a report to VAERS.
9. Public misperceptions about v-safe
Because v-safe was developed as a smartphone-based platform, there was some early confusion regarding whether participation required download of a mobile application (commonly referred to as an app). Inquiries received at CDC from participants indicated that some thought they would be automatically enrolled by their vaccination site or that v-safe would automatically capture their vaccine information from other immunization information systems. To address these misconceptions, CDC incorporated clarifying content into public presentations and v-safe information pages available online at cdc.gov/vsafe. Another common misconception by the general public is that v-safe can be used as a vaccine passport or that the unique QR code embedded in each account can be used to verify vaccination status. V-safe was never intended to serve as a digital certification of vaccination; the QR code in each account only encodes each registrant’s unique v-safe identifier and does not encode any information related to vaccination.
10. Implications for future versions of v-safe and development of similar systems
Lessons learned from the experience of the earliest days of the v-safe program may be helpful for development of similar systems. Entry of new participants was most robust in the first six months of vaccine distribution, which may indicate that healthcare workers and those who sought vaccines at the earliest opportunity were more willing to participate in v-safe than the general population. As enrollment numbers grew, we expanded computing resources beyond their initial scale to accommodate large data downloads. The sheer size of the dataset requires that analysts incorporate efficient coding practices to successfully complete analytic tasks requiring integration of multiple files from this large, linked dataset. Currently, data are housed on a dedicated server with multiple drives allowing for up to 2 terabytes storage per drive.
The free text fields included in each health survey have posed some challenges analytically. Surveys completed on days 0–7 after vaccination ask participants to record any other symptoms or health events beyond the solicited local and systemic reactions captured in the first week, while surveys completed at day 14 and later ask participants to record any new symptoms or health events. Early reviews of the text fields indicated that while most participants used these fields as intended, some included content unrelated to the capture of symptoms or health events. Approximately 5% of health surveys overall contain free text entries but the number of surveys completed to date precludes individual review of each entry. We are currently utilizing natural language processing techniques to evaluate patterns of responses in these fields and planning to explore the utility of coding of symptoms reported in free text fields to the Medical Dictionary for Regulatory Activities (MedDRA) terms.
Finally, although the v-safe platform itself has performed reliably with minimal disruptions, text messaging to v-safe participants relies on external factors beyond the platform. These factors include passage through carrier networks that may employ algorithms to filter out spam messaging and individual level phone settings that may be configured to block text messages from unknown sources. One potential solution to this filtering is short code messaging, used to send high volumes of text messages from registered 5- and 6-digit phone numbers that are less likely to be flagged as spam. We hope to implement short code messaging at a future date. The time required to obtain approval for short code usage can be lengthy, and we suggest that sufficient lead time be incorporated into planning for similar systems.
11. Summary
The v-safe after vaccination health checker platform was launched in December 2020 as part of CDC’s COVID-19 pandemic response and as a new approach to active vaccine safety surveillance [6]. During the first year of an evolving vaccination program, the platform provided valuable early safety information for COVID-19 vaccines authorized and administered in the United States. Analysis of v-safe data has contributed to characterization of the overall safety profile of these COVID-19 vaccines. Findings have provided confirmation to vaccine providers and recipients that the overall reactogenicity profiles following COVID-19 vaccination are similar in the broader population to those observed during clinical trials [6], [7], [8], [9], [10], [11], [12], [13], [14] in a more carefully controlled study population. Because the v-safe survey reminders prompt people to participate and report how they are feeling after vaccination, even when they don’t have a symptom or health event to report, we have been able to capture the full range of experiences post-vaccination. In addition, the v-safe platform has supported the COVID-19 vaccine pregnancy registry by identifying potentially eligible participants and facilitating enrollment to support critical safety monitoring in a population that was not included in clinical trial assessments of COVID-19 vaccines [5], [22].
This new active surveillance system is well-positioned to characterize common adverse events and health impacts experienced by recipients of new vaccines and assist in the capture of more serious events in VAERS via the v-safe call center. V-safe supplements CDC’s other vaccine safety surveillance systems (VAERS and the Vaccine Safety Datalink), both of which are useful for assessment of more rare and serious AEs, by capturing near real-time data describing local and systemic reactogenicity experienced after receipt of COVID-19 vaccine. Additionally, v-safe can identify special populations for further study, as demonstrated by the COVID-19 vaccine pregnancy registry. As COVID-19 vaccination recommendations continue to evolve, the v-safe platform must strike a balance between the agility required by an evolving pandemic response and the need to maintain operational stability of a platform with millions of participants. However, the platform has demonstrated ample ability to scale up as numbers of vaccine recipients have grown, and sufficient flexibility to incorporate additional features such as the ability to register dependents. These qualities of scalability and flexibility have been critical in the early days of safety monitoring of COVID-19 vaccines in the United States and suggest that the v-safe platform has promise as a potential application for future pandemic response activities requiring large-scale national vaccination programs, as well as for novel vaccine rollout activities in a non-pandemic context.
Author disclaimer: The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC).
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
All v-safe participants, COVID-19 vaccine providers and vaccination sites for supporting promotion of v-safe, CDC’s Vaccine Task Force.
Funding Sources
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Nonfinancial technical support to build and maintain the v-safe infrastructure for data capture and messaging to participants was provided from Oracle during the conduct of the study.
All authors attest they meet the ICMJE criteria for authorship.
Footnotes
Appendix ASupplementary data to this article can be found online at https://doi.org/10.1016/j.vaccine.2022.12.031.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
Supplementary data 1:
Click here to view.(71K, docx)
Data availability
v-safe data is available publicly at data.cdc.gov
References
1. Tran A., Witek T.J. The Emergency Use Authorization of pharmaceuticals: history and utility during the COVID-19 pandemic. Pharmaceut Med. 2021;35(4):203–213. [PMC free article] [PubMed] [Google Scholar]
2. Shimabukuro T.T., Nguyen M., Martin D., deStefano F. Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS) Vaccine. 2015;33:4398–4405. doi:10.1016/j.vaccine.2015.07.035. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
3. McNeil M.M., Gee J., Weintraub E.S., Belongia E.A., Lee G.M., Glanz J.M., et al. The Vaccine Safety Datalink: successes and challenges monitoring vaccine safety. Vaccine. 2014;32(42):5390–5398. [PMC free article] [PubMed] [Google Scholar]
4. U.S. Food & Drug Administration (2021, July 12). COVID-19 vaccine safety surveillance. U.S. Department of Health & Human Services. https://www.fda.gov/vaccines-blood-biologics/safety-availability-biologics/covid-19-vaccine-safety-surveillance#CMS Accessed Jun 3, 2022.
5. Shimabukuro T.T., Kim S.Y., Myers T.R., Moro P.L., Oduyebo T., Panagiotakopoulos L., et al. Preliminary findings of mRNA COVID-19 vaccine safety in pregnant persons. N Engl J Med. 2021;384(24):2273–2282. [PMC free article] [PubMed] [Google Scholar]
6. Gee J., Marquez P., Su J., Calvert G.M., Liu R., Myers T., et al. First month of COVID-19 vaccine safety monitoring – United States, December 14, 2020 – January 13, 2021. MMWR. 2021;70(8):283–288. [PMC free article] [PubMed] [Google Scholar]
7. Shay D.K., Gee J., Su J.R., Myers T.R., Marquez P., Liu R., et al. Safety monitoring of the Janssen (Johnson & Johnson) COVID-19 vaccine- United States, March-April 2021. MMWR. 2021;70:680–684. doi:10.15585/mmwr.mm7018e2. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
8. Chapin-Bardales J., Gee J., Myers T. Reactogenicity following receipt of mRNA-based COVID-19 vaccines. JAMA. 2021;325:2201–2202. doi:10.18176/jiaci.0683. [PubMed] [CrossRef] [Google Scholar]
9. Rosenblum H.G., Gee J.M., Liu R., Marquez P.L., Zhang B., Strid P., et al. Safety monitoring of mRNA vaccines administered during the initial 6 months of the US COVID-19 vaccination programme: an observational study of reports to the Vaccine Adverse Event Reporting System and v-safe. Lancet Infect Dis. 2022 doi:10.1016/S1473-3099(22)00054-8. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
10. Hause A.M., Baggs J., Marquez P., Myers T.R., Gee J., Su J.R., et al. COVID-19 vaccine safety in children aged 5–11 years – United States, November 3 – December 19, 2021. MMWR. 2021;70(5152):1755–1760. [PMC free article] [PubMed] [Google Scholar]
11. Hause A.M., Gee J., Baggs J., Abara W.E., Marquez P., Thompson D., et al. COVID-19 vaccine safety in adolescents aged 12–17 years – United States, December 14, 2020-July 16, 2021. MMWR. 2021;70(31):1053–1058. [PMC free article] [PubMed] [Google Scholar]
12. Hause A.M., Baggs J., Gee J., Marquez P., Myers T.R., Shimabukuro T.T., et al. Safety monitoring of an additional dose of COVID-19 vaccine – United States, August 12-September 19, 2021. MMWR. 2021;70(39):1379–1384. [PMC free article] [PubMed] [Google Scholar]
13. Hause A.M., Baggs J., Marquez P., Myers T.R., Su J.R., Blanc P.G., et al. Safety monitoring of COVID-19 vaccine booster doses among adults – United States, September 22, 2021 – February 6, 2022. MMWR. 2022;71(7):249–254. [PMC free article] [PubMed] [Google Scholar]
14. Hause A.M., Baggs J., Marquez P., Abara W.E., Olubajo B., Myers T.R., et al. Safety monitoring of COVID-19 vaccine booster doses among persons aged 12–17 years – United States, December 9, 2021 – February 20, 2022. MMWR. 2022;71(9):347–351. [PMC free article] [PubMed] [Google Scholar]
15. Gomez A.M. V-Safe: How everyday people help the CDC track COVID vaccine safety with their phones. Kaiser. Health News. 2021 https://khn.org/news/article/v-safe-how-everyday-people-help-the-cdc-track-covid-vaccine-safety-with-their-phones/ Accessed: Jun 3, 2022. [Google Scholar]
16. @Aloe9678. “I’ve been surprised at how many people don’t know about V-safe. I kind of like having them check on me.” [Twitter post; Mar 25, 2021]. Retrieved from https://twitter.com/Aloe9678/status/1374943058450640896 Accessed: Jun 3, 2022.
17. @hsofia. “I love the v-safe post-vaccination check-in. They keep reminding me that I can opt out any time, and I’m like NO! CHECK UP ON E FOREVER; I LIKE THIS SOLICITUDE! (… but yeah, maybe mention your lack of appetite)”. [Twitter post; Jun 23, 2021]. Retrieved from https://twitter.com/morinotsuma/status/1407551207237947393 Accessed: Jun 3, 2022.
18. @jsingpubhealth. “Surround yourself with people that check in on you like V-Safe” [Twitter post; Apr 9, 2021]. Retrieved from https://twitter.com/jsingpubhealth/status/1380676640267624451 Accessed: Jun 3, 2022.
19. @jzbetweeting. “I’m going to miss conversing with the V-safe robot, it feels like it really gets me.” [Twitter post; May 18, 2021]. Retrieved from https://twitter.com/jzbetweeting/status/1394770084489424904 Accessed Jun 3, 2022.
20. @alecperkins. “The daily “v-safe” check-ins from the CDC are pretty wild, too, seeing the government actually care about your well being.” [Twitter post: Apr 3, 2021]. Retrieved from https://twitter.com/alecperkins/status/1378693987658305539 Accessed Jun 3, 2022.
21. Tompkins L.K., Baggs J., Myers T.R., Gee J.M., Marquez P.L., Kennedy S.B., et al. Association between history of SARS-CoV-2 infection and severe systemic adverse events after mRNA COVID-19 vaccination among U.S. adults. Vaccine. 2022;40(52):7653–7659. [PMC free article] [PubMed] [Google Scholar]
22. Zauche L.H., Wallace B., Smoots A.N., Olson C.K., Oduyebo T., Kim S.Y., et al. Receipt of COVID-19 vaccine and risk of spontaneous abortion. NEJM. 2021 doi:10.1056/NEJMc2113891. [CrossRef] [Google Scholar]
23. Perrin, A. Mobile technology and home broadband 2021. Pew Research Center. https://www.pewresearch.org/internet/2021/06/03/mobile-technology-and-home-broadband-2021/ Accessed: Jun 3, 2022.
Articles from Vaccine are provided here courtesy of Elsevier
