Software crash bedeviled insurance exchanges

Delays on Health Insurance Marketplace weren't just from high traffic

WASHINGTON — The technical problems that have hampered enrollment in the online health insurance  exchanges resulted from the failure of a major software component, designed by private contractors, that crashed under the weight of millions of users last week, federal officials said Monday.

Todd Park, President Obama's top technology adviser, said the failure occurred in the part of the Web site that lets people create user accounts at the beginning of the insurance sign-up process. The crash prevented many people from viewing any of their insurance options or gaining access to information on what federal subsidies might be available.

"At lower volumes, it would work fine," Park said of the Web site, healthcare.gov. "At higher volumes, it has problems."

“Right now,” he added, “we’ve got what we think we need. The contractors have sent reinforcements. They are working 24-7. We just wish there was more time in a day.”

In some cases, the Web site does not recognize users who established accounts before Oct. 1, when the online marketplaces opened for consumers to shop for insurance. Other users are prevented from establishing accounts. Some who successfully established a marketplace account received an e-mail asking them to verify their e-mail addresses, but the link provided did not work.

The identification of the software component as the main cause of the Web site’s problems was the most detailed explanation that federal officials have given since the online marketplaces opened. The officials also rejected mounting criticism about the Web site’s overall design, saying that the rest of the site appears ready to handle the large volume of traffic.

But because of the initial failures, other parts of the complex system have yet to be proved under the intense strain of real-world conditions. And outside experts said that White House officials should have spent more time tending to the computer code and technology of the Web site, rather than recruiting Hollywood celebrities to promote it.

“It’s poorly designed,” said Luke Chung, the president of a database company in Virginia who has publicly criticized the site in recent days. “People higher up are given the excuse that there are too many users. That’s a convenient excuse for the managers to pass up the chain.”

Those comments echoed similar criticism on sites across the Internet, where Web designers and developers speculated about the reasons for the ongoing problems at the Web site, healthcare.gov. One discussion on the popular Web site reddit.com was titled "How not to optimize a website."

White House officials declined to identify the private contractors who had built the account creation function, citing a decision to keep that information private. They said the contractors had moved that part of the new system to beefed-up hardware and were busy rewriting the software code to make it more robust and efficient.

In the past week, wait times have dropped by half, officials said.

Officials said they had also added staff members at call centers to provide customers an alternative to the online system. The Web site currently says that people “in a hurry” can apply faster at a government call center using a toll-free telephone number, (800) 318-2596. But an operator at the call center said Monday that he could not help because he, too, was “experiencing technical difficulties with the Web site.”

Aneesh Chopra, who preceded Mr. Park as the federal government’s chief technology officer and helped create an earlier version of healthcare.gov, said he was confident that the system would be working effectively in the coming weeks.

Chopra noted that when United Airlines and Continental merged their online reservations systems, it took weeks to iron out problems.

“This is par for the course for large-scale I.T. projects,” Chopra said. “We wish we could launch bug-free, but in reality that’s not that easy to do. The reality is that if you have a product that people want, people will tolerate glitches because they expect them.”

Administration officials said they had relied heavily on contractors to build and operate the federal exchanges, under supervision of the Centers for Medicare and Medicaid Services. In the weeks before the marketplaces opened for business, those contractors expressed high optimism that their computer systems would work.

The prime contractor for the federal exchange — CGI Federal, a unit of the CGI Group, based in Montreal — and the company operating a “data services hub” for the government — Quality Software Services Inc., a unit of the UnitedHealth Group — told Congress at a hearing on Sept. 10 that they were ready for a surge of users when enrollment opened on Oct. 1.

But in recent days, officials at the companies declined to answer questions about the Web site’s problems. Linda F. Odorisio, a spokeswoman for CGI, and Matthew H. Stearns, a spokesman for UnitedHealth, refused last week and again on Monday to answer questions about their companies’ performance. Both companies said they had passed operational readiness reviews conducted last month by the federal government.

The Government Accountability Office, an investigative arm of Congress, said that CGI had received $88 million for work on the federal exchange through March, while Quality Software Services had received $55 million for work on the data hub. The hub allows exchanges to get inform,ation about a person’s income and citizenship from the Internal Revenue Service, the Department of Homeland Security and other agencies.

As the engineers for the contractors struggle to recover from the Web site’s failures, officials said, the partial shutdown of the federal government is also hampering efforts to carry out Obama’s health care law and has slowed work on a federal insurance marketplace for residents of more than 30 states.

All insurers participating in the federal exchange have been assigned an account manager, who serves as the primary point of contact with the exchange. The account manager is supposed to assist insurers, clarifying their responsibilities and answering questions about the federal Web site, enrollment transactions and other operational matters.

But many of the account managers have been furloughed in the shutdown.

The Obama administration has drafted a manual describing operations of the federal exchange, including the enrollment process. But federal officials said the shutdown had delayed a final review of the manual by lawyers and other federal employees who have been furloughed.

Joanne Peters, a spokeswoman for the Department of Health and Human Services, said that work being done on the Web site, in response to “overwhelming demand,” had begun to show results, reducing wait times and allowing more people to enroll.

“But,” she said, “we won’t stop until the doors to healthcare.gov are wide open.”