Making the Grade: Evaluating the Performance of State Health Insurance Marketplaces
The health insurance marketplaces created by the Affordable Care Act (ACA) are a key element of the Obama administration’s efforts to expand access to affordable health coverage. Prior to the passage of the ACA in 2010, many people who purchased their own health insurance encountered high costs and limited benefits; others were excluded altogether because insurers either refused to cover subscribers’ pre-existing conditions or charged much higher prices to individuals with serious medical conditions.1 To remedy this situation, the ACA directed each state to create a health insurance marketplace (or “exchange”) to increase consumers’ ability to choose among competing health plans. The goal was to provide consumers with “regulated yet competitive health insurance markets” where “insurers offer structured choices, compete on the basis of the price and quality of their services and must accept all applicants.”2
(a) Nevada and Oregon, for example, sought additional federal support for their SBMs after a disastrous implementation experience in the first open enrollment period. Idaho and New Mexico were late adopters who lacked adequate time to develop their own SBMs before the initial open enrollment period.
The ACA did not create one unified national program model, but instead established a framework for states to follow their own implementation paths. States could choose from three different approaches when establishing their marketplaces: a state-based marketplace (SBM), a federally-facilitated marketplace (FFM), or a state-partnership marketplace (SPM). The models varied on several key dimensions, including the source of funding, the division of responsibilities for program administration, and the flexibility surrounding program implementation.
The structure of the SBMs and FFMs is fairly straightforward, with either state (SBM) or federal (FFM) officials responsible for creating and maintaining online insurance marketplaces, managing enrollment, certifying eligibility for subsidies, reviewing qualified health plans, and assisting consumers through outreach and education efforts. SPMs are a hybrid, ceding control over the online marketplace, eligibility determinations, and enrollment to federal officials while allowing states to retaining control over plan management and/or consumer assistance programs.
A fourth marketplace option evolved over time: the federally-supported state-based marketplace (FSSBM), currently used in Idaho, Nevada, New Mexico, and Oregon.3 SPMs and FSSBMs both represent hybrid approaches to ACA implementation, but for different reasons. States that opted to create SPMs decided in advance to retain certain responsibilities, while FSSBMs emerged in response to implementation challenges facing SBMs – federal officials, in effect, tossed a lifeline to rescue troubled state-based marketplaces plagued by technical problems.(a) In both FSSBMs and SPMs, the federal HealthCare.gov portal determines eligibility for coverage and subsidies and manages the enrollment process.3
Despite their use of HealthCare.gov, FSSBMs are grouped together with SBMs in our analysis because these states originally chose to create a state-based marketplace.
A little over half of states use the federal marketplace, Healthcare.gov; 13 states, including Rhode Island, run their own state-based marketplace; and the remainder of states use one of the other exchange models.4 Rhode Island’s state-based marketplace, HealthSourceRI (HSRI), was created by an executive order from Governor Lincoln Chafee in 2011 and launched along with the other state and federal healthcare exchanges in 2013.5
TABLE 1: HEALTH INSURANCE MARKETPLACE MODELS

Kaiser Family Foundation (2016).4
Changes at HealthSourceRI
(b) Our report card focuses exclusively on the performance of the health insurance marketplaces. It does not attempt to measure the overall performance of the health care system in each state, the impact of the ACA on the cost of health insurance, the impact of state choices regarding Medicaid expansion, or the overall proportion of individuals in each state who remain uninsured. These factors are beyond the scope of our analysis.
Three years after the marketplaces launched, the critical question for policymakers is how well these different marketplace models have performed so far. To help answer this question, we created a report card that uses four key measures to grade the performance of all 51 marketplaces through the end of the second open enrollment period, which concluded in March 2015 (enrollment data for the 2016 open enrollment period is not yet available).
Our methodology is adapted from well-established models used to evaluate the performance of K-12 education policies,6 and our preliminary framework for the report cards was published in the Journal of the American Medical Association in 2015.7 As far as we know, ours is the first analysis to compare the outcomes of different marketplace types in all fifty states and the District of Columbia.
Our report card offers insights into how well the health insurance marketplaces fulfilled the ACA’s goals of expanding access to health insurance and controlling the cost of coverage by increasing competition.(b) We analyze the results both by state and by type of exchange, and benchmark HealthSourceRI’s results against other New England states and the rest of the country.
(c) All data in our report is through the end of the second open enrollment period that concluded in early 2015. Data for the third open enrollment period, which ran from November 1, 2015 through January 31, 2016, will not be available until the spring of 2016.
(d) This indicator measures “effectuated” enrollment, as it not only reflects consumer willingness to select a plan during open enrollment, but also captures how many individuals actually paid premiums to their insurer to finalize their coverage.
(e) Since most insurers offer more than one QHP, the number of issuers (rather than the number of plans) offers a practical indicator of competition among different insurers for eligible enrollees.
HOW WE EVALUATE EXCHANGE PERFORMANCE
Our report card grades the performance of health insurance marketplaces on their two core functions: facilitating enrollment and increasing competitiveness.(c)
First, marketplaces must enroll eligible individuals. As the former director of the Massachusetts Health Connector wrote in 2010, “Simply put, an exchange sells insurance. If it does not enroll many people, it has failed at its core mission.”10 We gave states two grades to evaluate how well they accomplish this critical task. The first is Enrollment Growth: the percentage change, from 2014 to 2015, in the number of individuals selecting a marketplace plan during the open enrollment period.11 The second grade is Market Penetration: the percentage of total eligible participants who signed up and paid for(d) a marketplace plan in 2015.12 Taken together, these grades offer two distinct yet complementary measures of a marketplace’s success in enrolling individuals in health insurance plans.
The second core function of a marketplace is to promote “a competitive insurance market in which consumers can make rational decisions.”13 Competition not only improves consumer choice, it helps keep prices in check. To measure the competitiveness of each state’s marketplace, we again used two measures of performance. First, we ranked states by Premium Cost: the percentage change in premiums from 2014 to 2015.14 Since Silver level health insurance plans are the most commonly purchased, we focused on the price of the lowest-cost Silver plan in each marketplace. The second measure we used to evaluate competitiveness was Plan Choice: the percentage change, from 2014 to 2015, in the number of insurers(e) offering qualified health plans (QHPs) to consumers through each marketplace.15 Increasing the number of insurers selling health plans through the marketplace encourages carriers to limit the rate of premium increases and offer a wider range of products.
Types of Health Insurance Plans
FIG. 1: EVALUATING STATE HEALTH INSURANCE MARKETPLACES: OVERALL PERFORMANCE

Hackey, Overall Marketplace Performance, 2015
TABLE 2: RHODE ISLAND’S SCORE

(f) Other measures paint a more favorable picture of Rhode Island’s implementation of the ACA. For example, at the end of 2014, only 6% of the state’s non-elderly population was uninsured, the second lowest rate in the nation. However, since the proportion of uninsured individuals also reflects a state’s efforts to enroll individuals in Medicaid, we did not use this in our report card, which is meant to evaluate marketplace performance. In particular, this measure reflects different policy choices by state governments about whether or not to expand Medicaid eligibility, which may be unrelated to the performance of a state’s health exchange.
STATE REPORT CARD RESULTS
Each state received a letter grade from A to F for each of the core indicators outlined in the previous section, as well as a combined GPA that reflects its overall performance. Maine was the nation’s top performer, receiving the only grade in the A range, an A-, while Hawaii placed last as the only state to receive an F. The national average was a C+.
Rhode Island’s state-based marketplace earned an overall score of B-, placing it in the top half of states in terms of overall marketplace performance. Rhode Island did well in promoting competition in the individual insurance market: it was one of only five states to earn an A on both limiting premium growth and expanding the number of qualified health plans available in the marketplace. However, Rhode Island ranked poorly in signing up new enrollees and performed just slightly above the national average in the proportion of eligible individuals enrolled.(f)
Enrollment: Enrollment Growth
Nationally, marketplace enrollment increased 46% in 2015, and 36 states experienced year-over-year growth of 50% or more. Rhode Island performed well on this measure during the first open enrollment period, as 28,485 individuals selected a marketplace plan for 2014 through HSRI. During the second open enrollment period, however, it lagged behind other states in New England and recorded the fourth lowest rate of enrollment growth (10%) in the nation. Rhode Island received a grade of D on our report card for enrollment growth.
When discussing percentage growth, it is important to look at the base from which a state grew. In 2015, for example, Massachusetts led the nation with enrollment growth of 343%. Yet this extraordinary performance is in large part the result of a particularly weak showing in the first enrollment period, during which Massachusetts’ website was virtually inoperable for extended periods of time. After this rocky start, the Bay State redoubled its efforts and succeeded in increasing enrollment in 2015. Rhode Island, on the other hand, performed well during the first open enrollment period, leaving less room for growth in the second enrollment period.
However, unlike Rhode Island, some states that started out strong during the first enrollment continued to add large numbers of consumers during the second enrollment period. Florida’s federally facilitated marketplace, for example, enrolled the second highest number of individuals during the first open enrollment period and still managed to increase enrollment by 62% in the following year (compared to 10% in Rhode Island). Connecticut also performed relatively well during the first year and still increased its enrollment by 39% from 2014 to 2015.
Our results suggest that the type of marketplace may be connected to performance in this category. Of the nine states that received an A for enrollment growth, six were federally-facilitated marketplaces, two were state-based marketplaces, and one was a state-partnership marketplace.
FIG. 2: EVALUATING STATE HEALTH INSURANCE MARKETPLACES: ENROLLMENT GROWTH

U.S. Department of Health and Human Services (2014, 2015)11
FIG. 3: EVALUATING STATE HEALTH INSURANCE MARKETPLACES: MARKET PENETRATION

Kaiser Family Foundation (2015)
Enrollment: Market Penetration
On average, states enrolled only 34% of potential participants in an “effectuated” marketplace health plan (i.e., the individual signed up for a plan during the open enrollment period and paid their first monthly premium to the insurer). Only three states signed up more than half of potential enrollees in 2015, with Vermont leading the nation at 52%. HealthSourceRI performed slightly above average nationally, enrolling 38% of eligible enrollees, but lagged behind other states in New England (with the exception of Connecticut).
Competition: Premium Cost
Across the nation, the average premium for the lowest cost Silver plan rose 2.9% from 2014 to 2015. Sixteen states recorded decreases in cost, while five experienced double-digit increases. Rhode Island ranked sixth in the nation with a 10.9% reduction in premiums from 2014 to 2015. It outperformed several other New England states, with the exception of New Hampshire’s top-scoring state-partnership marketplace.
FIG. 4: EVALUATING STATE HEALTH INSURANCE MARKETPLACES: PREMIUM COST

Holahan, Blumberg, and Wengle (2015)
(g) In the fall of 2015, however, the CEO of UnitedHealth Group warned that if its financial losses for ACA products continued, the company would stop offering plans in some marketplaces.22
Competition: Plan Choice
A majority of states increased the number of issuers offering qualified health plans (QHPs) on their exchanges from 2014 to 2015, with some states adding as many as five new insurers. Three states lost one participating insurer, while in the remainder of states the number of insurers stayed the same. Once again, the nation’s top performing marketplace in this category was New Hampshire, which jumped from one to five insurers – a 400% increase – from 2014 to 2015. The number of health insurers offering coverage through HealthSourceRI increased from two in 2014 – Blue Cross and Blue Shield of Rhode Island and Neighborhood Health Plan of RI – to three in 2015 with the addition of United HealthCare.(g) This 50% increase enabled Rhode Island to earn an A in this category.
FIG. 5: EVALUATING STATE HEALTH INSURANCE MARKETPLACES: PLAN CHOICE

Kaiser Family Foundation (2016)15
WHICH EXCHANGE MODEL WORKS BEST?
Before the ACA took effect, many supporters of health care reform believed that state-based marketplaces, while potentially more expensive, would be more effective. As Harvey Fineberg, a physician and public health expert who was formerly the president of the Institute of Medicine, noted in 2012, “state based insurance exchanges will have the potential to introduce and oversee clearer consumer information and more appropriate competition among insurers.”16 Governor Lincoln Chafee’s executive order creating HealthSourceRI reflected this assumption, touting local control as the key to a better performing and more responsive marketplace.17
Our analysis, however, suggests that state-based marketplaces did not outperform their federally-facilitated and state-partnership peers. Contrary to expectations, SBMs earned the lowest average GPA (2.13) of any marketplace type on our report card. Furthermore, nearly half of SBMs (46%) ranked in the bottom 20% of states in terms of their overall GPA. In contrast, federally-facilitated marketplaces had an average GPA of 2.46, while state-partnership marketplaces were in the middle with a GPA of 2.18. (In our analysis, FSSBMs are grouped together with SBMs, despite their use of HealthCare.gov, because these states originally intended to create a state-based marketplace). The ten best-performing states in our report card included seven FFMs, one SBM (Massachusetts), one FSSBM (New Mexico), and one SPM (New Hampshire).
REVIEWING RHODE ISLAND’S PERFORMANCE
Overall, Rhode Island earned slightly above-average grades for the performance of its health care exchange in the first two open enrollment periods. The state excelled in improving the competitiveness of its health insurance market by recording one of the largest reductions in premiums in the U.S. (-10.5%) and attracting new insurers to the marketplace. This success, however, does not mean that a state-based marketplace necessarily outperforms other models. Several states with federally-facilitated marketplaces, such as Indiana, Maine, and Mississippi, also earned top scores in marketplace competitiveness while spending a fraction of what Rhode Island did to implement its state-based marketplace.7 A state-based marketplace, in short, is not the only way to improve marketplace competition.
On the other primary component of our evaluation – enrollment – Rhode Island’s performance was mixed. On the positive side, the state was among the top twenty performers in the country in terms of market penetration. It enrolled 38% of eligible enrollees, slightly above the national average. However, despite spending more than $150 million on exchange implementation, including an intensive television and radio advertising campaign and the establishment of local consumer assistance centers, Rhode Island failed to enroll a majority of eligible individuals for coverage (though only three states were able to achieve this in 2015). More importantly, five federally-facilitated marketplaces and one state-partnership marketplace were able to enroll a larger share of eligible individuals than Rhode Island using HealthCare.gov, rather than a state-based platform.
Rhode Island was less successful in increasing enrollment from 2014 to 2015, with growth at a meager 10%. HSRI enrollment increased from 28,485 in 2014 to 32,253 in 2015, out of a total pool of 86,000 potential marketplace enrollees in 2015.12 Higher-than-average market penetration numbers indicate that the state was able to enroll a large share of participants in the first enrollment period, but it did not build on this momentum in the second enrollment period. Rhode Island might seek lessons from other New England states with higher rates of enrollment growth, such as Connecticut (39%), Maine (69%), and New Hampshire (32%), as well as state-based marketplaces in the District of Columbia and Maryland and the state-partnership marketplace in Delaware, which all logged enrollment growth of 75%. Rhode Island’s performance when it comes to enrollment suggests that spending more for a state-based exchange does not guarantee enrollment success.
(h) This assessment is shared even by staunch supporters of the ACA. As the Kaiser Family Foundation’s Larry Levitt argued, “If enrollment plateaus, we may see growing discussion of whether the law is fulfilling expectations in covering the uninsured.”24
(i) Despite enrolling more than one million customers in marketplace plans around the nation, Aetna CEO Mark Bertolini expressed “serious concerns about the sustainability of the public exchanges.”25 Meanwhile, UnitedHealth Group reported losses of $475 million on ACA marketplace plans in 2015.25
Going forward, increasing the proportion of enrollees represents a significant challenge for Rhode Island, just as it does for the rest of the country. The remaining population of uninsured individuals is a difficult target market. To date, studies suggest that enrollees purchasing health insurance through public exchanges of all types tend to be older than uninsured and other insured adults.18 It will be important to persuade younger and healthier customers to enter the marketplace – even though, for many middle-income individuals, paying the penalty for not having health insurance continues to be a cheaper option than purchasing insurance.19
Early results from the 2016 open enrollment period show that only 26% of individuals who signed up for coverage through December 2015 were between the ages of 18 and 34, while actuaries argue that this demographic group should represent 40% of enrollees to ensure the long term viability of plans.20 The continued under-representation of younger, healthier enrollees, regardless of exchange type, contributes to insurers’ growing losses on ACA plans. Several of the nation’s largest insurers – each with demonstrated records of managing utilization and health care costs – are now reevaluating their continued participation in the health care marketplaces.21
In the wake of significant budget cuts and substantial reductions in staffing, HealthSourceRI will need to find new, more cost-effective ways to market itself to hard-to-convince customers. Stagnant enrollment – particularly in a small market such as Rhode Island’s – could lead insurers to withdraw from the market, limiting competition among remaining marketplace plans.(h) For example, UnitedHealth Group’s CEO recently warned that the company “can’t subsidize a market that doesn’t appear at this point to be sustaining itself.”22 Following this announcement, other large national insurers expressed similar concerns about the long-term viability of the risk pool for health insurance marketplaces of all types.(i)
THE PATH FORWARD
In the long run, the success of HealthSourceRI depends on its ability to do more with less. Funding is unlikely to return to previous levels, when federal grants provided generous support for outreach and education. It will be essential to find creative ways to support and expand the exchange within a budget that is currently around $11 million per year. Since Rhode Island is already a top-performer in fostering marketplace competition, increasing enrollment could be a higher priority. To do so, HealthSourceRI may need to dedicate additional resources to bolster consumer outreach and customer service.
In this new fiscal climate, it will be important to identify ways to minimize the overhead costs of operating the marketplace. Rhode Island could free up funds for activities that support enrollment growth by reducing the cost of operating HSRI’s web portal. Although this limits prospects for systems integration with other state agencies, the state could avoid the high cost of future technology upgrades. Several state-based marketplaces now use the HealthCare.gov platform as federally-supported state-based marketplaces (FSSBMs).
Alternatively, Rhode Island could collaborate with other states in New England to create a regional marketplace. As Jim Wadleigh, the CEO of Connecticut’s exchange noted after the U.S. Supreme Court’s decision in King v. Burwell, “We would love to do something regional on the individual side or the small business side that would bring economies of scale.”23 Rhode Island could also continue to operate its own state-based marketplace but focus intensely on cutting costs.
Whatever path Rhode Island chooses, a key focus should be marketing the exchange to younger, healthier customers. Though strengthening customer service and outreach efforts to achieve this is a challenge in the context of limited resources, the long-term viability of a small individual health insurance market such as Rhode Island’s depends on broadening the risk pool and enrolling a higher proportion of eligible individuals.
ADDITIONAL INFO
-
Endnotes
- Hackey, Robert. (2000). “The Politics of Reform.” Journal of Health Politics, Policy and Law, 25(1): 211-23.
- Kingsdale, Jon. (2014). “After the False Start – What Can We Expect from the New Health Insurance Marketplaces?” New England Journal of Medicine, 370(5): 393-96.
- Giovannelli, Justin and Kevin Lucia. (2015).The Experiences of State-Run Marketplaces that Use HealthCare.gov. Washington, DC: The Commonwealth Fund.
- Kaiser Family Foundation. (2016). “State Health Insurance Marketplace Types, 2016.” [Data files].
- Mulligan, Jessica. (2015). “Choosing a Health Exchange for Rhode Island,” Footnote, June 17.
- StudentsFirst. (2013). State of Education: State Policy Report Card – 2013. Sacramento, CA.
- Hackey, Robert B. and Erika L. May. (2015). “Measuring the Performance of State Health Insurance Marketplaces.” Journal of the American Medical Association, 314(7): 667-668.
- Salit, Richard. (2015). “As HealthSourceRI Budget Shrinks, Customers Left on Hold.” Providence Journal, October 29. Salit, Richard. (2015). “R.I. Healthcare Innovation Group Recommends Creating New Office of Health Policy to Curb Spending.” Providence Journal, December 1.
- Mach, Annie and C. Stephen Redhead. (2014). Federal Funding for Health Insurance Exchanges. Washington, DC: Congressional Research Service.
- Kingsdale, Jon. (2010). “Health Insurance Exchanges – Key Link in a Better-Value Chain.” New England Journal of Medicine, 362: 2147-2150.
- Enrollment Growth data is from enrollment figures published by the U.S. Department of Health and Human Services at the end of the first and second open enrollment periods. Office of the Assistant Secretary for Planning and Evaluation. (2014). Health Insurance Marketplace: Summary Enrollment Report for the Initial Annual Open Enrollment Period.Washington, DC: US Department of Health and Human Services. Office of the Assistant Secretary for Planning and Evaluation. (2015). Health Insurance Marketplaces: 2015 Open Enrollment Period March Enrollment Report. Washington, DC: US Department of Health and Human Services.
- Market Penetration data is from the Kaiser Family Foundation’s analysis of effectuated enrollment in each marketplace as of September 30, 2015. Kaiser Family Foundation. (2015). “Marketplace Enrollment as a Share of the Potential Marketplace Population.” [Data files].
- Aaron, Henry and Kevin Lucia. (2013). “Only the Beginning – What’s Next at the Health Insurance Exchanges?” New England Journal of Medicine, 369(13): 1185-87.
- The data and methodology for calculating Premium Cost were drawn from: Holahan, John, Linda J. Blumberg, and Erik Wengle. (2015). Marketplace Premium Changes Throughout the United States, 2014-2015. Washington, DC: The Urban Institute.
- Plan Choice data is from the Kaiser Family Foundation’s analysis of the number of insurers issuing QHPs in each marketplace in 2014 and 2015. Kaiser Family Foundation. (2016). “Number of Issuers Participating in the Individual Health Insurance Marketplaces.” [Data files].
- Fineberg, Harvey. (2012). “A Successful and Sustainable Health Care System – How to Get There from Here.” New England Journal of Medicine, 366 (11): 1020-27.
- Rhode Island Executive Order No. 11-09. (2011). Establishment of the Rhode Island Health Benefits Exchange. September 19.
- Blavin, Fredric, Michael Karpman, and Stephen Zuckerman. (2016). “Understanding Characteristics of Likely Marketplace Enrollees and How They Choose Plans.” Health Affairs, 25(3): 535-39.
- Goodnaugh, Abby. (2016). “Many See I.R.S. Penalties as More Affordable Than Insurance.” New York Times. January 3.
- Armour, Stephanie. (2016). “Young Adult Health Insurance Sign-Ups Disappoint.” Wall Street Journal. January 8.
- Wilde Mathews, Anna. (2016). “Insurers Under Pressure to Improve Margins on Health Plans.” Wall Street Journal. February 10.
- Wilde Mathews, Anna, and Stephanie Armour. (2015). “Biggest Insurer Threatens to Abandon Health Law.” Wall Street Journal. November 19.
- Armour, Stephanie. (2015). “States Eye Health Exchange Options.” Wall Street Journal. June 26.
- Alonso-Zaldivar, Ricardo. (2015). “Is Obama’s health overhaul losing steam?” Associated Press. October 15.
- Wilde Mathews, Anna and Anne Steele. (2016).“Aetna Reports Surge in Profit and a Dark Spot on Results.” Wall Street Journal. February 1.
-
Original Questions
What will be the impact on Rhode Island if the Affordable Care Act is overturned? What are the other options for affordable health care? Would RI continue to be served by the Federal exchange vs. a state operated exchange and at what cost?
-
Type of Research
Context Insights
- Explores the questions of Policy Leaders by providing a unique academic perspective to the research project
- Provides context for challenging state issues