Earlier this year Mission Hospital of Asheville was named one of the “50 Best Hospitals” in the United States. That award, given by Healthgrades, a for-profit hospital rating company, in effect designated HCA-owned Mission as the best hospital in North Carolina. No other hospital in the state was listed.
Mission has touted this award in its marketing materials, citing it as proof of the high performance of the hospital’s staff and management.
“We are grateful to Healthgrades for their independent evaluation of our clinical excellence,” Chad Patrick, CEO of Mission Hospital, said in a news release announcing the award.
Healthgrades is owned by Red Ventures, a giant media company based in Indian Land, South Carolina, that also owns Bankrate, The Points Guy (a ratings system for airline mileage programs and credit cards), CNET, and the Lonely Planet travel guides.
According to an article this year by The Verge, “Red Ventures’ business model is straightforward and explicit: It publishes content designed to rank highly in Google search for “high-intent” queries and then monetizes that traffic with lucrative affiliate links.”
Red Ventures is one of several organizations that publish consumer-facing evaluations of hospitals, and whose ratings are used by Mission — and other local hospitals — in their marketing materials.
Because hospital raters like Healthgrades charge hospitals a fee to use their ratings in their marketing materials, the ratings are “more a marketing ploy than a way to evaluate performance,” Paul Keckley, a Nashville-based healthcare policy analyst and industry expert, who produces an industry newsletter called The Keckley Report, told Asheville Watchdog.
Keckley called the ratings unhelpful for consumers. “It’s almost buyer beware,” he said.
Leapfrog, another organization whose ratings are touted by Mission, is a nonprofit founded and supported by an employer coalition. It focuses on patient safety and issues semi-annual grades for hospitals, like a report card’s A, B, C, D and F. It has given Mission an “A” for much of the last few years under HCA.
Mission also received a “Best Regional Hospitals” award from US News & World Report for 2022-2023, though it places 17th in the magazine’s North Carolina list of regional bests.
Healthgrades gave AdventHealth Hendersonville two awards in 2023 and 2021 for “Patient Safety Excellence” and “Outstanding Patient Experience.” It also named UNC Health Pardee — formerly Margaret R. Pardee Memorial Hospital in Hendersonville — one of “America’s 100 Best Hospitals” in 2023 and 2022.
Leapfrog also gave AdventHealth Hendersonville an “A” and Pardee a “B” in its latest ratings.
Rating the Raters
Such consumer-facing ratings are meant to inform patients of where to go for the best care. But medical experts doubt they truly inform patients.
In a 2019 article, the New England Journal of Medicine’s NEJM Catalyst turned the tables. The authors of the article, “Rating the Raters: An Evaluation of Publicly Reported Hospital Quality Rating Systems,” found that the various hospital rankings and ratings — often producing different results — can be confusing for consumers.
“The numerous currently available public hospital quality rating systems frequently offer conflicting results, which may mislead stakeholders relying on the ratings to identify top-performing hospitals,” according to the report (paid subscription required).
Of the four groups analyzed by the New England Journal of Medicine, U.S. News & World Report got the highest grade, a B — none of the ratings companies got an A — followed by the government’s own rating system, Hospital Compare by the Centers for Medicaid & Medicare Services (CMS) with a C.
Mission Hospital, located in Asheville, North Carolina, is Mission Health’s flagship hospital, licensed for 815 beds. Healthgrades named Mission Hospital as of one America’s 50 Best Hospitals from 2020-2023, identifying the top 1% of hospitals in the nation for providing the highest clinical quality year over year. Leapfrog Hospital Safety Grade recognized Mission Hospital with an “A,” nationally recognizing Mission Hospital’s achievements in protecting patients and providing safer healthcare.— From Mission Hospital’s website
Healthgrades and Leapfrog received the lowest grades. Leapfrog, which awarded Mission Hospital an “A,” itself got a C- in the 2019 NEJM ratings. Healthgrades, which named Mission the highest-rated hospital in North Carolina, received the lowest grade, a D+.
Overall, ratings systems should be used cautiously, the NEJM report said, as “they likely often misclassify hospital performance and mislead.” The report noted that insurance companies use the ratings data to steer patients to certain facilities, and hospitals use the data as a way to identify areas for improvement.
They also noted that publishing companies like private equity-backed Red Ventures perform these ratings with a profit motive.
“[H]aving hospitals pay the rating systems to be able to display their performance or to allow use of their ratings for hospital marketing or advertising — may create unfortunate incentives. Specifically, there is a concern that the business of selling these ratings leads to a model that encourages multiple rating systems to intentionally identify different ‘best hospitals.’ ” the authors wrote.
“Close to Meaningless”
“I think they’re [the hospital ratings] close to meaningless,” said Dr. Eric Thomas, a nationally known patient safety expert who is a professor of medicine at the McGovern Medical School at the University of Texas Health Science Center in Houston.
“It’s really not very helpful to use these kinds of measures to rank hospitals on safety because it just doesn’t give you a full picture of safety,” Thomas said.
Thomas said that the safety measures that go into the scores do not find enough of the harmful events that can occur in a hospital, and “there are a huge number of safety events that harm patients,” he said. A study of the safety monitoring system for Medicare beneficiaries a few years ago, he said, detected only about 2 percent of the safety events.
“A hospital could be performing very well on a limited number of safety measures,” Thomas said, “yet at the same time, people could be coming into that hospital every day and being harmed by horrible errors.”
Complicating the ratings, Thomas said, is that hospitals may focus only on what is publicly reported instead of “focusing on the things that could potentially improve safety across the board, like nurse staffing.”
Unionized nurses at Mission have repeatedly protested in the streets about the safety risks they say are posed for patients because of inadequate staffing, a topic previously reported on by Asheville Watchdog.
Legislation to limit the number of patients for each registered nurse is now under consideration in Congress. “Numerous studies have shown that safe nurse-to-patient staffing ratios result in higher quality care for patients, lower healthcare costs, and an overall better workplace for nurses,” said Rep. Jan Schakowsky (D- IL) said in a press release when the legislation was re-introduced March 30.
“Staffing is a way of addressing all the safety problems that are happening in a hospital at the same time,” Thomas said. “What you need to do is create a culture and train clinicians to have the ability to communicate well as teams, to identify risky situations and be able to respond appropriately in those moments.”
“And that gets into the bigger issue of the safety culture of an organization,” said Thomas.
Dr. Matt Austin, associate professor of anesthesiology and critical care medicine at the Johns Hopkins University School of Medicine, said that the ratings systems have divergent results because they look at different aspects of care. He is a paid consultant for Leapfrog through his work for the Johns Hopkins Armstrong Institute for Patient Safety and Quality, and advises on the scientific methodology behind Leapfrog’s safety grades.
Terms like best hospital don’t tell you a lot
Austin said that the hospital ratings attempt to boil down complicated and multi-factor care in a hospital into a single assessment. “So using terms like ‘best hospital’ doesn’t really tell you a whole lot,” he said.
Because the rating organizations look at different things, Austin said, “I think consumers, unfortunately, do have to look under the hood a little to really understand what each rating system is trained to tell them.”
The Hospital Safety Grade’s methodology is peer‐reviewed and fully transparent, according to Leapfrog, and is available to the public at www.hospitalsafetygrade.org/your-hospitals-safety-grade.
Austin said the ratings “have utility as a jumping off point for conversations” with one’s physician. While hospital choice can be limited to where a patient’s physician has admitting privileges, he said, “You try to educate yourself and make good decisions.”
“Am I going in for bypass surgery? I’m probably going to dig into how the hospital does on that versus other hospitals,” Austin said. It really doesn’t matter to a heart patient that the hospital is highly rated for knee replacements, he said.
Healthgrades looks strictly at clinical outcomes — Did the patient die? How many patients have complications from common conditions and procedures, and have to be readmitted? — using data from Medicare. It evaluates nearly 4,500 hospitals a year, or nearly three quarters of the 2022 total of 6,093 hospitals in the United States.
The results of its evaluations are compared with its proprietary risk methodology, according to Dr. Brad Bowman, Healthgrades Chief Medical Officer and Head of Data Science.
Healthgrades has also awarded top ratings to Mission since 2021, including a 2022 “Patient Safety Excellence Award,” though notably Mission did not win the 2023 Healthgrades award for “Outstanding Patient Experience.”
Bowman told Asheville Watchdog in an email that while there are other strong hospitals in North Carolina, Mission was recognized by Healthgrades for the eighth year in a row “as one America’s 50 Best Hospitals — the only one in North Carolina.”
Bowman declined to say how much it charges hospitals such as Mission for a licensing agreement to use its ratings in marketing materials. Healthgrades also does not make its full methodology and risk adjustment models public, said other medical researchers.
US News said in its 2022-2023 rankings that Mission rated as “high performing” in eight adult health procedures, including colon cancer surgery, heart bypass surgery, heart attacks, heart failure, strokes, hip and knee replacements, and chronic obstructive pulmonary disease (COPD). Multiple other procedures scored “average” and three scored “below average,” including ovarian cancer surgery, hip fracture, and transcatheter aortic valve replacement.
US News evaluates more than 5,000 hospitals. Its ratings rely most heavily on clinical outcomes, such as survival rate, using Medicare data, but it also uses data on the nurse-to-patients ratio that reflects the quality of care, and conducts a survey of physicians, according to its website. It also includes patient experience data from Medicare, but this accounts for just 5 percent of the score.
Leapfrog uses 30 different measures in five dimensions, Leah Binder, the CEO of Leapfrog, told Asheville Watchdog. It looks at such things as complications and infections after surgery, pressure ulcers, practices to prevent errors, and medications safety. It will be adding data on nurse-to-patient ratios later this year, Binder said.
“Safety should come first,” Binder said, “because nothing else matters if you’re not safe in the hospital, no matter how good your surgeon is.”
“Hospitals are complex and dangerous places” where “some horrible things happen,” Binder said. “Sadly, it is a very common problem in this country.”
Binder said that a study by BMJ, the weekly medical journal of the British Medical Association, suggested that hospital errors, injuries, and accidents are “the third leading cause of death in the United States, with 250,000 people a year dying.” Others don’t die, she said, but they “just get egregiously harmed.”
The dangers of hospitals rose last year, according to a recent report by The Joint Commission, which accredits hospitals. It said that reports of adverse events, called sentinel events, that killed or severely harmed patients rose to 1,441 in 2022, a 19 percent increase from 2021.
Leapfrog graded nearly 2,900 hospitals last fall, or close to half of the total hospitals in the nation, and generally gives about 35 percent of those an “A” rating, Austin said. “It’s a fairly generous grading curve,” he said.
Hospitals rated “A” are more likely to pay the ratings companies to license use of the ratings in marketing material.
Patient experience differs from patient safety
Patient safety measures, such as death and complications, predominate for the hospital ratings, but Leapfrog integrates some patient experience measures from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS), a survey conducted by the federal Centers of Medicare and Medicaid Services (CMS).
Leapfrog uses the HCAHPS data on nurse communication with patients, doctor communication with patients, the responsiveness of hospital staff, and communications about medications, Austin told Asheville Watchdog.
Mission’s scores by Leapfrog on the responsiveness of hospital staff, communications with nurses and communication about medications are all in red (indicating poor), with communication with doctors in yellow. “So, they don’t actually do all that great,” he said.
Medicare’s website includes a Hospital Compare function, and Mission Hospital scores just two stars out of five for patient experience, though it rates four stars overall. The fours stars rating is based on the performance of the hospital in treating heart attacks and pneumonia, general safety, and its readmission rates, according to Hospital Compare.
The two-star rating for patient experience “would be a big red flag,” said Carol Cronin, executive director of the Informed Patient Institute. That rating “says there’s a problem,” she said.
The breadth of the problem is detailed in Mission’s scores for the two-star patient experience. Mission scores lower than the national average on every one of the 10 issues covered in the HCAHPS survey. Some were notably larger deviations from the national averages, according to information from CMS.
Only 46 percent of Mission patients reported that they “always” got help as soon as they wanted, compared to a national average of 66 percent. Only 53 percent of patients reported that their room and bathroom were “always” clean, compared to 72 percent of the national average. And only 50 percent of patients reported that they would definitely recommend the hospital, compared to a national average of 70 percent.
“Do I feel safe?”
The rating organizations emphasize patient safety, while paying less attention to the patient experience.
Thomas said patient experience is an “evolving topic.” It was once assumed that patients thought of safety in the same way medical professionals do, by looking at the possibility of harmful events, he said. But“when you talk to patients and families and ask them in a more open ended way what does safety mean to you, it’s a lot about emotional safety. Do I feel safe?”
This sense of safety is about how hospital staff are communicating with the patient and treating the patient with respect, said Thomas. “Right now, there are no patient-centered measurements. All the surveys don’t really get at that (emotional safety) very directly.”
“You could have a successful surgery in terms of no complications, but the experience could have been miserable for all kinds of reasons,” Thomas said.
“Rankings are not based on patient experiences,” said Caitlin Donovan, a senior director at the National Patient Advocacy Foundation in Washington, DC. The rankings “should be looked at with great suspicion,” she said.
Cost is not covered in ratings
Keckley also said prices would be a key factor in how a consumer evaluates a hospital, but costs are not addressed in any ratings. In a January Gallup poll, 38 percent of Americans said they delayed medical treatment in 2022 because of cost. Another poll last May by HealthSparq found that 44 percent of respondents avoided getting care in the last year because they were not sure about costs, and 68 percent of respondents said they’d be more likely to book a medical appointment if they knew the estimated out of pocket costs.
In 2023, for the second year in a row, Mission was out of compliance with a federal rule on price transparency, the Patient Rights Advocate, a national nonprofit focused on price transparency, reported in February. The report was endorsed by North Carolina Treasurer Dale Folwell.
Nancy Lindell, Division Director for Public and Media Relations for Mission, said the hospital has complied by providing on its website “a consumer-friendly Patient Payment Estimator tool” that helps patients understand their out-of-pocket costs, as required by the federal regulation.
Inflated prices are a key issue in the two federal antitrust lawsuits that Mission faces. One of those lawsuits was filed by the cities of Asheville and Brevard, as well as the counties of Buncombe and Madison, and the other was filed by private citizens and local businesses.
Mission’s pricing was also part of a report released April 11 by the Community Engagement Council for Transylvania Regional Hospital and sent to Sam Hazen, the CEO of HCA. It said HCA had “broken the trust and lost the confidence of this community.”
The new report used an example of exorbitant pricing for a shoulder arthroscopy from the original Brevard antitrust lawsuit: “The Mission-Asheville price for this procedure was $2,419 — nearly three times the statewide average of $897.”
Kaiser Healthcare News researchers zeroed in on trauma-team activation charges in 2020, and found that for North Carolina HCA Mission charged an average of $9,187 compared to the non-HCA fee of $5,175.
At the end of the day, said Donovan of the National Patient Advocacy Foundation, “There is no correlation between what a hospital charges and the value of what was done.”
Asheville Watchdog is a nonprofit news team producing stories that matter to Asheville and Buncombe County. Barbara Durr is a former correspondent for The Financial Times of London. Contact her at email@example.com.