PROVIDENCE, R.I. [Brown University] — A five-star rating is supposed to be a gold standard and a reliable measurement of quality. Yet when it comes to Medicare Advantage health care plans, highest-star ratings don’t always indicate top-level performance for all beneficiaries, a study by Brown University researchers found.
The new analysis, published in JAMA Health Forum, shows that five-star ratings, the scale’s highest, are only modestly associated with quality of health care experience for racial/ethnic minorities and socioeconomically disadvantaged enrollees in the plans.
“It's clear from this study that the way the current star ratings are constructed, they’re not capturing the full experience of all beneficiaries,” said David Meyers, lead study author and a Brown assistant professor of health services, policy and practice.
Meyers said that studies in other health care contexts have found that rating systems don't tend to address racial and socioeconomic inequities within those systems without an explicit plan to do so.
“And in this case, the fact that Medicare Advantage plans can earn higher-star ratings overall even if they’re not adequately serving minority beneficiaries doesn’t provide much of an incentive to address health equity,” he said.
Since 2008, the U.S. Centers for Medicare and Medicaid Services (CMS) has used a five-star rating system to measure the performance of Medicare Advantage plans. Not only can potential enrollees use these star ratings to make decisions about plans, but highly rated plans stand to reap large financial benefits.
“Payment bonuses for four- and five-star plans can top $6 billion annually, so there’s enormous financial incentive for these plans to perform well in terms of rating,” Meyers said.
More than one-third of Medicare beneficiaries were enrolled in Medicare Advantage plans in 2019. Yet there hasn’t been much research done to validate what the star ratings mean and if they do an adequate job of capturing the experiences of an enrollee who might be in a plan, Meyers said. In addition, these plans enroll higher proportions of racial/ethnic minorities and people with lower income and education than the traditional Medicare program.
Meyers and his team wanted to find out how well the star rating system works for different populations of enrollees. Their study addresses the association between a Medicare Advantage plan’s overall star rating and what the star rating would have been if calculated for enrollees who are racial/ethnic minorities or who have lower socioeconomic status, as well as whether plans with higher star ratings have lower disparities in care.
The Medicare Advantage star ratings are generated using data from all enrollees in a plan, and CMS assigns bonus payments without segmenting results by race, ethnicity, or socioeconomic status, Meyers said.
For the study, the researchers analyzed the experiences of 1,578,564 Medicare Advantage enrollees in 2015 and 2016, taking into consideration racial/ethnic and socioeconomic factors. They compiled data from five different sources that allowed them to recreate about 75% of the measures that go into the star rating calculation. From that individual data, Meyers said, they were able to reconstruct CMS’s star ratings, but broken down by race, ethnicity and socioeconomic status.
“What we essentially did was to go through and say, ‘If we were to rate a plan based only on the experience of Black beneficiaries — or Hispanic beneficiaries, or low socioeconomic beneficiaries — what star rating would the plan get?’” Meyers said.
The researchers then compared the different star ratings by group to see how they related to each other. The study had several key findings.
First, the researchers observed only a modest correlation of simulated star ratings when calculated for low vs. high socioeconomic status enrollees, and between racial/ethnic minority and white enrollees in the same plan. Second, plans with higher ratings have larger racial/ethnic disparities than did those with lower ratings. Third, contracts with lower concentrations of low socioeconomic status and Black/Hispanic individuals had larger disparities and worse quality for these individuals. By contrast, plans with higher enrollments of low socioeconomic status and Black/Hispanic individuals did a better job for these beneficiaries.