President Barack Obama seemed to lose control of his second-term agenda even before he was sworn in, when a school massacre led him to lift gun control to the fore. Now, as he tries to pivot from a stinging defeat on that issue and push forward on others, the president finds himself rocked by multiple controversies that are demoralizing his allies, emboldening his political foes and posing huge distractions for all.

It’s unclear how long he will be dogged by inquiries into last year’s deadly attack in Libya, the IRS targeting of tea party groups and now the seizure of Associated Press phone records in a leak investigation. But if nothing else, these episodes give new confidence and swagger to Republicans who were discouraged by Obama’s re-election and their inability to block tax hikes as part of the Jan. 1 “fiscal cliff” deal.

Taken together, these matters will make it harder for the administration to focus on its priorities — racking up a few more accomplishments before next year’s national elections.

“It’s a torrential downpour, and it’s happening at the worst possible time, because the window is closing” on opportunities to accomplish things before the 2014 campaigns, said Matt Bennett, who worked in the Clinton White House. From here on, he said, “it’s going to be very, very difficult.”

So far, there’s no evidence that Obama knew about — let alone was involved in — the government actions in question. But a president usually is held accountable for his administration’s actions, and Republicans now have material to fuel accusations and congressional hearings that they hope will embarrass him, erode his credibility and bolster their argument that his government is overreaching. Even some of his Democratic allies are publicly expressing dismay at the AP phone records seizure.

Obama advisers on Tuesday cast the trio of controversies as matters that flare up in an institution as complex as the U.S. government, and they questioned the impact of them. The one exception, advisers said, was the brewing scandal at the Internal Revenue Service, which they see as the issue most likely to strike a chord with Americans.

The IRS has apologized for what it calls “inappropriate” targeting of conservative political groups, including tea party affiliates, that were seeking tax-exempt status in recent years. Attorney General Eric Holder said Tuesday he had ordered a Justice Department investigation.

But Holder distanced himself from the decision to subpoena the AP records, saying he had no part in it, stepping aside because he had been interviewed in a government investigation into who provided information for a news story that disclosed details of a CIA operation in Yemen.

The press case sparked bipartisan outcry, with several GOP and Democratic officials questioning Holder’s department’s actions in the matter. Republican National Committee Chairman Reince Priebus said the attorney general should resign over the issue, adding: “Freedom of the press is an essential right in a free society.”

Despite the noisy controversies, White House advisers tamped down suggestions that Obama would make any sudden moves, such as firing top officials or shaking up his team. Aides said they want more details from an inspector general report on IRS actions before deciding how to proceed on that issue.

On all three matters, the White House Tuesday steered blame to other administration agencies. The disputed Benghazi talking points, advisers said, were chiefly the CIA’s work. In discussing the IRS controversy, the White House has emphasized the agency’s independent status.