In 1997 one research team said that the value of the ratings were limited because at that time public medical data was difficult to access, and a lack of quality data limited the usefulness of any ratings.
In 2001 a research paper said that US News ratings were based on medical school assessments, counts of research publications, student opinion surveys, and counts of faculty. The paper noted that the rankings were broadly accepted, cited, and used to make decisions by all sorts of stakeholders. The public image of the rankings was that they were unbiased.
A 2005 study considered US News ratings with "Hospital Compare", which is a rating published by Centers for Medicare and Medicaid Services. That study found that the two ratings systems frequently made different recommendations for the best hospitals.
A 2009 study said that when a hospital's place in US News rankings changed, then consequently there could be as much as a 5% change in certain kinds of patient admissions to that hospital which might be attributed to that change in rankings.
A 2010 study criticized the US News rankings for not disclosing the relationship between their ranking methodology and the reputation of any given hospital. The study stated that US News rankings matched other rankings about the market reputation of hospital brands, and alleged that US News rankings are overly influenced by brand image while doing less to assign a rank by health outcome metrics of patients who use medical services in those hospitals.
A 2012 study compared the US News rankings with hospital rankings at Consumer Reports. That study said that the two systems ranked hospitals differently, with 8% of graded hospitals having a similar position in each other's rankings. In larger urban areas with multiple hospitals, the two ranking systems made different recommendations in 81% of cases.