The Obama administration told senators it didn’t notify Congress about the pending swap of Sgt. Bowe Bergdahl for five Taliban officials because of intelligence the Taliban might kill him if the deal was made public.

That fear — not just the stated concerns that Bergdahl’s health might be failing — drove the administration to quickly make the deal to rescue him, bypassing the law that members of Congress be notified when detainees are released from the U.S. prison at Guantanamo Bay, Cuba, congressional and administration officials said Thursday.

They spoke only on condition of anonymity because they were not authorized to comment publicly.

Since Bergdahl’s release on Saturday, administration officials including President Barack Obama, Defense Secretary Chuck Hagel and National Security Adviser Susan Rice have said publicly that the key reason for the secret prisoner swap was evidence that Bergdahl’s physical health was deteriorating after five years in captivity.

But on Wednesday night, administration officials told senators in a closed session that the primary concern was the death risk if the deal collapsed.

At a news conference in Brussels on Thursday, Obama said he makes no apologies for recovering Bergdahl, and he said the furor in Washington over the exchange has made the matter a “political football.” He appeared to be referring to potential danger to Bergdahl’s life when he said that “because of the nature of the folks that we were dealing with and the fragile nature of these negotiations, we felt it was important to go ahead and do what we did.”

There was no overt threat by the Taliban but rather an assessment based on intelligence reports that Bergdahl’s life would be in jeopardy if news of the talks got out and the deal failed, said two senior U.S. officials familiar with the efforts to free the soldier.

In public comments, State Department spokesman Marie Harf told reporters Thursday, “There were real concerns that if this were made public first, his physical security could be in danger.” The risks, she said, included “someone guarding him that possibly wouldn’t agree and could take harmful action against him. So as we needed to move quickly, all of these factors played into that.”

Not everyone in Congress was convinced.

“I don’t believe any of this,” said Sen. Lindsey Graham, R-S.C. “First, we had to do the prisoner deal because he was in imminent danger of dying. Well, they saw the video in January and they didn’t act until June. So that holds no water. Now the argument is the reason they couldn’t tell us is because it jeopardized his life. I don’t buy that for a moment because he was a very valuable asset to the Taliban.”

Bergdahl himself remained in a military hospital in Germany. His hometown of Hailey, Idaho, called off a big celebration planned for his eventual homecoming, citing security concerns.

Several administration and congressional officials said that a December video shown to senators in a briefing portrayed Bergdahl’s health as in decline but not so desperately that he required an emergency rescue. An assessment by U.S. intelligence agencies about the video in January came to the same conclusion, said two congressional officials familiar with it.

Still, the administration continued to cite the health issue. Obama said, “We had a prisoner of war whose health had deteriorated and we were deeply concerned about. And we saw an opportunity and we seized it.”

Hagel was referring in part to the threat from Bergdahl’s captors when he said Sunday that “there was a question about his safety,” administration officials told the senators in a closed-door briefing on Wednesday.

In that meeting, both Republican and Democratic senators complained that not even the chairman and ranking member of the intelligence committee, who are trusted with some of the nation’s most sensitive secrets, were notified of the agreement, said three congressional officials who were in the briefing. They spoke only on condition of anonymity because they were not authorized to discuss the matter publicly.

Sen. Angus King, an independent from Maine, said, “We were briefed that if these discussions had leaked out, there was a reasonable chance Bowe Bergdahl may have been killed. And that was one of the pieces of information that gave some credence as to why it had to be kept quiet.”

Taliban fighters freed Bergdahl on Saturday and turned him over to a U.S. special operations team in eastern Afghanistan. Under the deal, five Taliban militants were released from Guantanamo and flown to Qatar, where they are to remain for a year under conditions that have not been spelled out in public.

A federal law requires Congress to be told 30 days before a prisoner is released from Guantanamo, but Obama administration officials said it did not apply in what they deemed an emergency situation.

A senior administration official, speaking on condition of anonymity because the official wasn’t authorized to discuss the matter publicly, said that “senators were told, separate and apart from Sgt. Bergdahl’s apparent deterioration in health, that we had both specific and general indications that Sgt. Bergdahl ‘s recovery — and potentially his life — could be jeopardized if the detainee exchange proceedings were disclosed or derailed.”

Sen. Saxby Chambliss, R-Ga., said after the briefing Wednesday night that Bergdahl appeared drugged but not at imminent risk of death.