Narrative Overview and Discussion
Back in February, I devoted a lot of my time to helping people get vaccine appointments, but that phase has now passed. I’ve taken some time over the past few days to check on the vaccination status of my constituents.
I did an email survey from May 10 through 12 that reached roughly 4000 people in my district – Back Bay, Fenway, Allston, Brighton, Watertown, and Belmont. Among the first 2042 who responded, 98% reported having had at least one vaccination dose.
That sounded high, so I sent a follow up email pleading, “I’d be grateful if you could take the survey even if you have not been vaccinated.” Another 559 responded, but the vaccination rate was almost the same among them.
In the responding sample, all age groups and all zip codes had vaccination rates over 96%. People on my email list include a wide range of political views, but all are engaged and pay attention to the news. Among people fitting that description, it appears that almost all have been able to get vaccinated.
I did a reality check against the state’s zip code vaccination report which comes out weekly. In the zip codes that I represent, the share of people who have gone through the vaccination process and are showing up in the state’s vaccination counts is very high – over three fourths in all age groups except those under 19. In the Fenway area, the vaccination rate is reduced by large populations of students under 19.
I asked people about their experience in scheduling an appointment. As I expected based on the phone calls and emails I was getting, many had a bad experience in February and March. Among those who scheduled in that period, one quarter said their experience was a ”nightmare.” In February and March, only 44% reported a smooth scheduling experience. The “nightmare” number was down to 11% in April. By May, 71% said their experience was “smooth” and 19% reported a “minor hassle,” while 10% continued to experience a “nightmare.”
Virtually everyone, 99% regardless of when they scheduled their appointment, reported a satisfactory experience at the vaccination sites – “safe and fast.”
After vaccination, 80% said that they felt safer, but were still following health care guidance: “I feel much safer and I am getting out more, but I am still wearing a mask and following all recommended procedures.” Much smaller groups erred towards too cautious or too care-free: 5.6% said that they do not feel much safer and 3.7 % said that they were fully going back to normal.
Overall, I was comforted to confirm that in my district, after a rough start, the vaccination system has been reaching people. That confirmation is consistent with the state’s good rankings as compared to other states on various metrics.
There are some less affluent and less engaged areas of the state where the numbers are weaker. Both the Governor and the legislature are giving those areas a lot of attention with outreach to build awareness and make vaccination as convenient as possible.
The other group I am concerned about are those who have lost faith in the health care system and do not trust the advice to get vaccinated. I believe that is a very small group in my district. It was 1.2% of those who responded to my survey and based on the state’s high reported vaccination rates, it cannot be much larger than that in my district. But I do feel concerned for the vaccine hesitant.
I had a call with a group of vaccine hesitant people who had reached out to me and I got a feel for their wounded mistrust of the health care system. Most had had a bad experience that they attributed to health care errors and most preferred alternative medicine. I couldn’t win their trust back on the call and it was disappointing.
But I do take comfort that almost everyone I can reach is already benefiting from the success that our creative pharmaceutical industry has had in swiftly creating several vaccines that do save lives. We can be hopeful that the industry will be able to keep pace with emerging variants. From a public sector standpoint, the main challenge is to reach the communities where vaccination rates are still low.
Survey Methods and Limitations
We sent brief emails inviting people to complete a short survey. The messages went out in three parts:
- Using Mailchimp, we sent an email at 3:30PM on May 10 to approximately 3600 residents of my senate district — Back Bay, Fenway, Allston, Brighton, Watertown, and Belmont — and an additional approximately 500 people outside my district who also subscribe to email list. We closed that initial survey page after 24 hours.
- When I saw how high the initial reported vaccination rates were, I was afraid my sample might not be fully representative, so I decided to send a second email asking people to respond, even if they were not vaccinated. There was no tracking of who actually completed the survey form. However, our email tool, Mailchimp tracks when recipients open or click on emails. We sent a follow-up email at 7:30AM on May 12 to approximately 2000 people who did not show as having clicked on the original email. We closed that second survey page on May 13 at 11:50AM.
- In addition, the survey invitation was sent to 4 community news lists in my district on May 10 at 3:30PM. That survey page remained open for 24 hours. We do not know the number of people receiving or interacting with those messages.
The three groups of messages each brought people to a separate URL that fed the same form. As a result, we were able to track which of the three mailing groups each form response came from (but not specifically who responded).
The recipients had all affirmatively subscribed to my email list for office news (no politics) or to a community news list. So, the universe of respondents was self-selected and not random. Recipients may have forwarded the survey to others. Responses on other issues indicate that the people on my email list are politically diverse, although generally people who feel comfortable with my political views are likely over-represented. Certainly, people who are engaged and follow the news are over-represented on the list.
The survey was written so that if a person indicated that they did not vaccinated they were asked why. If a person indicated that they had been vaccinated they were asked about their experience.
| Message Group | Unique Deliveries | Form Link Clicks | Form Responses (% of deliveries) |
|---|---|---|---|
| Initial main Mailchimp mailing to all subscribers | 4143 | 2341 | 2042 (49%) |
| Follow up Mailchimp mailing to non-clickers | 2120 | 636 | 559 (26%) |
| Mailing to community lists | Unknown | Unknown | 142(?) |
| TOTAL | >4143 | >2977 | 2743(?) |
Survey Responses
Question 1: Have you received one or more doses of vaccine for COVID-19?
As the cross tabs below indicate, in this sample of respondents, vaccination was near universal and did not depend much on where people lived or their age group (except for those below 16). However, the sample does skew older, so young un-vaccinated people are underrepresented.
| Message Group | DECLINE | NO | YES | %YES |
|---|---|---|---|---|
| All subscribers | 4 | 36 | 2002 | 98.0% |
| Initial non-clickers | 7 | 8 | 544 | 97.3% |
| Community lists | 1 | 4 | 137 | 96.5% |
| TOTAL | 12 | 48 | 2683 | 97.8% |
| Zip | DECLINE | NO | YES | % YES |
|---|---|---|---|---|
| 02115 | 3 | 112 | 97.4% | |
| 02116 | 1 | 140 | 99.3% | |
| 02134 | 58 | 100.0% | ||
| 02135 | 2 | 6 | 435 | 98.2% |
| 02199 | 11 | 100.0% | ||
| 02215 | 4 | 96 | 96.0% | |
| 02472 | 1 | 14 | 668 | 97.8% |
| 02478 | 1 | 17 | 945 | 98.1% |
| Other | 1 | 2 | 207 | 98.6% |
| Total | 5 | 47 | 2672 | 98.1% |
| Age | DECLINE | NO | YES | %YES |
|---|---|---|---|---|
| 16 to 45 | 2 | 12 | 728 | 98.1% |
| 46 to 65 | 2 | 25 | 918 | 97.1% |
| Over 65 | 1 | 10 | 1020 | 98.9% |
| Under 16 | 1 | 0.0% |
Reality Check: Comparison of survey responses to vaccine reporting system results.
The state publishes weekly reports of vaccination by age and zip code. Except for in the decennial census, the census bureau estimates population by zip code only on an rough basis and with a delay so there is no fully accurate way to get vaccination rates by zip code. The table below presents that imprecise data; it includes some obviously wrong numbers for a couple of zip codes, but that reflects the roughness of the population data. The data generally show very high vaccination rates in older age groups across all zip codes, consistent with the survey results. The results do not vary much across zip codes when age mix is factored out. The youngest zip codes are 02115 and 02215 (see the census data).
Individuals with at least one dose — rates by zip code and age.
| 0-19 Years | 20-59 Years | 60-64 Years | 65 to 74 years | 75+ Years | All Ages | |
|---|---|---|---|---|---|---|
| 02115 | 16% | 68% | 97% | 92% | 97% | 55% |
| 02116 | 14% | 66% | 72% | 79% | 81% | 61% |
| 02134 | 19% | 82% | 71% | 104% | 91% | 77% |
| 02135 | 9% | 74% | 87% | 88% | 80% | 68% |
| 02199 | 0% | 79% | 331% | 57% | 123% | 81% |
| 02215 | 19% | 67% | 52% | 108% | 152% | 54% |
| 02472 | 9% | 75% | 65% | 95% | 88% | 66% |
| 02478 | 15% | 85% | 105% | 108% | 85% | 69% |
| Total | 14% | 74% | 78% | 93% | 87% | 64% |
See computations in this spreadsheet.
Question for those not vaccinated: If you have not received a COVID-19 vaccine, why not?
Among the very small group who had not been vaccinated, vaccine hesitancy was the most common reason — not difficulty in accessing vaccine.
| Reason for Non Vaccination | # not vaccinated | % of not vaccinated | % of full sample |
|---|---|---|---|
| Concerned about vaccine and may not get vaccinated. | 32 | 67% | 1.2% |
| Had difficulty arranging an appointment. | 7 | 15% | 0.3% |
| Under 16. | 1 | 2% | 0.0% |
| Not a priority. | 8 | 17% | 0.3% |
| Total | 48 | 100% | 1.7% |
Question 2: Where did you receive your first dose?
This sample was somewhat more reliant on the mass vaccination sites than the state as whole, but not too different. Compare the chart below to the Weekly COVID-19 Dashboard for May 13 at page 13.
| Where Vaccinated | Count | Share |
|---|---|---|
| Mass Vaccination Site | 709 | 26% |
| Pharmacy | 591 | 22% |
| Hospital | 542 | 20% |
| Other | 345 | 13% |
| Community Health Center | 247 | 9% |
| Medical Practice | 131 | 5% |
| Local Health Clinic | 118 | 4% |
| Total Responding to Question | 2683 | 100% |
Questions 3 and 4: When did you schedule your first appointment? Did you have difficulty scheduling an appointment?
Question 3 requested the date that the appointment was scheduled, as opposed to the date of the appointment itself. February was the month when the most people had trouble with scheduling as eligibility for vaccination opened up wider. More recently, supply has caught up with demand and scheduling appointments has been easier.
| When Scheduled | # Scheduled | Smooth | Minor Hassle | Nightmare |
|---|---|---|---|---|
| January | 248 | 69% | 19% | 12% |
| February | 663 | 44% | 31% | 25% |
| March | 872 | 44% | 33% | 23% |
| April | 841 | 46% | 43% | 11% |
| May | 21 | 71% | 19% | 10% |
Question 5: How was the experience where you got vaccinated??
People had good experiences generally with the actual vaccination experience, regardless of when they got vaccinated.
| When Scheduled | Scheduled | Safe and Fast — Satisfactory | Unsatisfactory for any reason | % Satisfactory |
|---|---|---|---|---|
| January | 252 | 249 | 3 | 99% |
| February | 657 | 645 | 12 | 98% |
| March | 864 | 853 | 11 | 99% |
| April | 844 | 834 | 10 | 99% |
| May | 21 | 21 | 100% |
Question 6: What best describes your lifestyle after being vaccinated?
Most people seem to be heeding the balanced guidance offered by public health authorities.
| Count | % | |
|---|---|---|
| I feel free and am going back to normal. | 100 | 3.7% |
| I feel much safer and I am getting out more, but I am still wearing a mask and following all recommended procedures. | 2156 | 80.4% |
| I do not feel much safer and I am still taking all precautions. | 150 | 5.6% |
| I was never very worried. | 42 | 1.6% |
| I am only partially vaccinated. | 203 | 7.6% |
| None of the above | 30 | 1.1% |
| Total Responding to Question | 2681 | 100.0% |
Good morning, dear Will.
As always, thank you for your work for all of us.
I very much appreciate.
I do think this new journey the country is starting on – the honor system re vaccinations – is off the walls.
Have a day filled with good.
Good morning, dear Will.
As always, thank you for your work for all of us.
I very much appreciate.
I do think this new journey the country is starting on – the honor system re vaccinations – is off the walls.
Have a day filled with good.
Excellent report, process. Thank you very much
Thanks Will. This is very helpful. There are two different herd immunities, one local and another national. While one immunity overlaps the other, it is reassuring that locally we are focused on science and safety. With the new CDC mask guidelines, there is reason to be excited about the summer.
Thank you and your team for being proactive and transparent.
Thank you for the reults of the survey
The website sign up was a disaster. Never should people have to compete for an appointment slot!
I would check everyday and find either no openings or they were too far away.
The pre-signup idea was a better idea but I did not use it.
I got lucky? I suppose because one of my healthcare providers contacted me and said they will have a vaccine soon and will let me know when it’s available. That’s what ended up working for me.
The state website was poorly thought out and the company it was farmed out to should be sued in my opinion.
I agree. It was a crazy time finding an appointment especially for those who were at higher risk but didn’t qualify as one of the 12 highest risk categories. I had been quarantined for 17 months and was aggravated with the process. I lucked out and found an appointment at the Mount Ida UMASS campus and the process was smooth after that. The only thing that I think could have been worked out better was the record transfer to my PCP. They had no way at my first dose to let my PCP know I had had my vaccine. I sent them a photo copy of my vaccine card but it would have been great to have an electronic link or something. The second time I went they had a link an a printable report that I could send to my doctor.
Indeed. A very rough start.
Thank you for putting this all together Will. You and your team are awesome.
I wasn’t contacted about making an appointment until the waiting list signup was made available. That was easily the best option as it was essentially set and forget and not nerve-wracking like I imagine the online appointment process was. It’s crazy to me that before the waiting list was available, it was like buying tickets for a popular concert or event, where you could get locked out of the system because too many people were hitting it, you may lose the appt you thought you had “in your cart”, etc. I do understand that it was cobbled together quickly, but I hope we can learn from this experience and have a better setup at the ready for the next time we need mass response for something.
Yes. And I feel we may all need boosters eventually, so the process may need to be repeated.
Thanks for this insight. I am impressed at the response levels! It’s uplifting to see this level of civic engagement.
Re: Vaccines: The initial website model was a disaster. Changing to a pre-registration system seems best, a push-out as people become eligible rather than a desperate chase.
Thank you for your hard work. Have a great weekend.
I received an email from Beth Israel Lahey Health in late February, as I qualified under the 2 health conditions requirement, so I never had to go through the ordeal of trying to schedule an appointment on my own.
I’m not sure we will ever get much below the 2% mark for the distrustful/ conspiracy minded/ alt-med crowd unfortunately. I have followed the Science-based Medicine movement since it’s inception, and it’s clear that the issue is not a knowledge deficit. We just have to make sure we contain their narratives, and avoid the spread of misinformation and conspiracy theories that originate in this population, that are then weaponized by forces that find them politically convenient.
Hi Kerry, how are you? Glad to hear your doing well.
I can second all of your points about the vaccine-hesitant crowd.
I only have one qualifying health condition. I received many emails from Beth Israel Lahey that I would be eligible soon and after I became eligible, the mails said, basically, “be patient, we are working on it and will let you know.” Meanwhile, I tried using the state web site and the unofficial “Are there any slots available” sites every day, for at least an hour. (Longer on Thursdays when new appointments became available at many sites.) It was pretty much a nightmare of wasted hours. Every few days it would promise something, but then I would go to the signup page and it would either tell me there was nothing available, or it would take all my information and then tell me there was nothing available, or it would put me on a wait list. On one occasion, the “wait time” on such a list started at 40 minutes, gradually ticked down to 20 minutes and then jumped to over 3,000 minutes! As a professional computer programmer, they web sites all seemed thoroughly inept.
Finally, about 3 weeks after I came eligible, there were several hundred appointments on several consecutive days about a week away in Lowell. Seemed a little far, but tenable. I signed up and it Worked! The next day, Beth Israel Lahey let me know they has appointments available, possibly closer and a day or two sooner, but I didn’t want to chance it and kept my Lowell Hospital appointment.
At the vaccine site, everything went very smoothly and quickly. It seemed very competent and well run, and the people were wonderful.
My only, very minor, issue was they could have had a large sign next to the parking lot entry so you could tell you were at the right place and where to turn. There were several entries to the parking lot, but some were closed off and others were exit-only. I ended up driving too far and had to double back. Fortunately, I was about 15 minutes early so no big deal. I’ve heard the signage at Gillette was very good, so I doubt this was an issue everywhere.
Thank you for this report. Glad you finally were let through.
Maybe we can “contain their narratives” with re-education camps?
Ummmm. No.
The biggest problem is our governor Screwup Chuck. Notice that he went for MassVax centers. The public chosen the pharmacy/hospital route. I think he secretly wants a Mass Bay Pandemic Authority model and wants to institutionalize the pandemic so he can appoint his pals to well paying sinecure positions. Imagine getting a seat on the Central Vaccination Planning staff or how about The Vaccination Advisory Board or even the Vaccination Management Control Board.
In February, I rcvd a text from BI to go to what I think might hv been a website, I then turned to my tech savvy niece, forwarded the incomprehensible (to me) text & she lined me up in 10 to 15 minutes…..no problems……. vax done at an empty supermarket with plenty of parking & it was a pretty smooth procedure with plenty of nurse types walking around to make sure no one fainted or otherwise had a bad reaction…..all in all, I was lucky, first with getting the text early on & second with having a tech savvy niece………
I believe that one of the reasons you got such a high percentage of responses to this survey Will is because you used the word “short” in your request for a response…….plus everyone likes & respects you……..as a friend of mine said about you, you are a true public servant Will.
This is super interesting! It is really nice to see at least someone in the government is doing their research and being very clear about their methodology. It also restores some of my faith in people in our community. After traveling to SC to visit family, I was feeling extremely jaded about people’s response to the pandemic and vaccinating.
I’ll also add that I was one of the people that had a nightmare experience in April. I was assigned a MassVax site in Foxborough, I live in Watertown, do not own a car, and noticed that it was still over an hour drive during work hours. There was no way that was happening.
When I qualified in early April (1 comorbidity) I ended up opening tabs to all the vaccination sites in the area, refreshing regularly, following a twitter update feed, looking up recommendations etc, to find an appointment. I only got one after refreshing the Tufts ZocDoc site 100+ times. That, of course, was not in Watertown, so I still had to spend ~$20 to get to Boston each way during work hours via rideshare apps. It would have been really nice if the sites in Watertown had been exclusive to locals. Or if MassVax had worked with pharmacies and offered those openings to locals rather than inconvenient places like Hynes and Foxborough.
In the end, I can afford it and my work is super flexible, but I also recognized that I am extremely privileged.
Thank you, Senator, for all your work — preparing, collecting and consolidating. I really appreciate all of this page’s content!!
I understand the difficulties experienced by many, and while it was difficult early on, I am (was) lucky enough not to urgently need vaccination. When April 5 (55+) rolled around, the vaccination sites machine was in full swing — got a spot on April 6 for April 6!
I do hope there is (has been) learning, as I fear we will need booster(s) and will encounter another supervirus in a few years (cat’s out of the bag, research-wise).
If there is (has been) learning, we will be much(!!!) better prepared for how to handle and how to outreach.
Thanks again for all you and your staff do, and have done with this great material. Big Thank You!!!
I’m glad so many responded and have been vaccinated. I’m thinking that those who are got vaccinated are more likely to respond to a survey like this than those who, for whatever reason, have not been vaccinated. That makes me wonder if the percentages reflect the kinds of folks who “do the right thing” rather than the public at large. Just thinking aloud.
I think public mask mandates should be enforced with fines. The fine should be a civil levy (about $5.00. Something like a parking ticket. You can appeal it, but the usual court filing fee would be higher than that (like $25.00). It still would make sense to do so in certain legitimate circumstances and you would preserve your civil liberties. The end result will be more compliance. We still have parking violators but are far fewer than parking compliance