What GAO Found
Under section 1115 of the Social Security Act, the Secretary of Health and Human Services (HHS) may approve Medicaid demonstrations to allow states to test new approaches to providing coverage and for delivering services that can transform large portions of states’ programs. However, GAO found that selected states’ evaluations of these demonstrations often had significant limitations that affected their usefulness in informing policy decisions. The limitations included gaps in reported evaluation results for important parts of the demonstrations. (See table.) These gaps resulted, in part, from HHS’s Centers for Medicare & Medicaid Services (CMS) requiring final, comprehensive evaluation reports after the expiration of the demonstrations rather than at the end of each 3- to 5-year demonstration cycle. CMS has taken a number of steps since 2014 to improve the quality of state-led evaluations, and in October 2017, officials stated that the agency planned to require final reports at the end of each demonstration cycle for all demonstrations. However, the agency has not established written procedures for implementing such requirements, which could allow for gaps to continue. CMS also plans to allow states to conduct less rigorous evaluations for certain types of demonstrations but has not established criteria defining under what conditions limited evaluations would be allowed.
Examples of Gaps in States’ Evaluations of Medicaid Section 1115 Demonstrations
Arizona
The state was required to evaluate whether providing long-term services and supports under a managed care delivery model improved access and quality of care. The evaluation report lacked information on important measures of access and quality.
Arkansas
The state was required to evaluate the effects of using Medicaid funds to purchase private insurance for more than 200,000 beneficiaries. The evaluation did not address a key hypothesis that using private insurance would improve continuity of coverage for these beneficiaries, who were expected to have frequent changes in income that could lead to coverage gaps.
Massachusetts
The state was required to evaluate the effectiveness of its approach of providing up to $690 million in incentive payments to seven hospitals to improve quality of care and reduce per capita costs. Evaluation reports submitted after 5 years provided no conclusions on the impact of the payments in these areas.
Source: GAO. | GAO-18-220
Federal evaluations led by CMS have also been limited due to data challenges that have affected the progress and scope of the work. For example, delays obtaining data directly from states, among other things, led CMS to considerably reduce the scope of a large, multi-state evaluation, which was initiated in 2014 to examine the impact of state demonstrations in four policy areas deemed to be federal priorities. Though CMS has made progress in obtaining needed data, it is uncertain when results from the multi-state and other federal evaluations will be available to policymakers because CMS has no policy for making results public. By not making these results public in a timely manner, CMS is missing an opportunity to inform important federal and state policy discussions.
Why GAO Did This Study
Demonstrations—which represented roughly a third of the more than $300 billion in federal Medicaid spending in 2015—are a powerful tool to test new approaches to providing coverage and delivering Medicaid services that could reduce costs and improve beneficiaries’ outcomes. Evaluations are essential to determining whether demonstrations are having their intended effects. States are required to evaluate their demonstrations and CMS can initiate its own federal evaluations of demonstrations.
GAO was asked to examine evaluations of demonstrations, including how the results have been used to inform Medicaid policy. This report examines (1) state-led evaluations and (2) federal evaluations. GAO reviewed evaluation documentation for eight states with high demonstration expenditures that varied in the number of years their demonstrations had been in effect and by geography. GAO also reviewed documentation for the ongoing federal evaluations and interviewed state and federal Medicaid officials. GAO assessed evaluation practices against federal standards for internal control and leading evaluation guidelines.
What GAO Recommends
GAO recommends that CMS: (1) establish written procedures for requiring final evaluation reports at the end of each demonstration cycle, (2) issue criteria for when it will allow limited evaluations of demonstrations, and (3) establish a policy for publicly releasing findings from federal evaluations of demonstrations. HHS concurred with these recommendations.
For more information, contact Katherine Iritani at (202) 512-7114 or iritanik@gao.gov.