A Republican senator said Wednesday he would block President Barack Obama’s nominees for Federal Reserve chairman and Homeland Security chief in a dispute over last year’s attack in Benghazi, Libya.

Sen. Lindsey Graham of South Carolina said using his Senate prerogative to stall all future nominees is his only leverage as Republicans try to force the administration to let survivors of the assault on the U.S. diplomatic mission talk to members of Congress.

“What I am asking for is to talk to the people who were there,” Graham told reporters at a Capitol Hill news conference, insisting that congressional investigator have access to survivors.

A diplomatic security agent who was a witness to the Sept. 11, 2012, raid that killed U.S. Ambassador Chris Stevens and three other Americans already has been deposed by the House Oversight and Government Reform Committee.

Graham said that occurred only because the individual was subpoenaed.

Benghazi is the rallying cry for conservatives who argue that the administration tried to mislead the American people in the heat of a presidential campaign by playing down a terrorist attack on Obama’s watch. In the months since, congressional Republicans have accused the administration of stonewalling their investigations.

In a letter to Graham, the State Department said it was concerned about congressional interviews with the survivors of the attack because of Justice Department said they could be witnesses in a criminal trial and any interviews outside the criminal justice process could jeopardize a case.

The department also wrote that “because these survivors are potential witnesses in a terrorism prosecution, as well as law enforcement professionals who engage in security activities around the world including at high-threat posts, disclosure of their identities could put their lives, as well as those of their families and the people they protect, at increased risk.”

Republicans who joined Graham at the news conference, including Sen. Kelly Ayotte, who served as New Hampshire’s attorney general, dismissed that argument from the department. Republicans said they would neither compromise national security nor jeopardize any prosecution.

Nominees in limbo are Janet Yellen, Obama’s choice to replace Ben Bernanke as chairman of the Federal Reserve, and Jeh Johnson, tapped to fill the vacancy at Homeland Security after Janet Napolitano resigned as secretary. The Senate also must decide the fate of judicial nominees as well as military officers.

Graham said he planned to vote for Johnson, but this was his only recourse.

Graham’s news conference came as a new poll showed the senator’s support plunging among Republican voters in South Carolina. Benghazi is an issue that energizes core Republicans.

The Winthrop Poll showed Graham with the backing of 45 percent of Republican voters in October, down from 72 percent in February. The poll conducted Oct. 19-27 interviewed 887 adults and had a margin of error of 3 percent.

Graham, who is seeking a third term next year, faces primary challenges from three candidates: Nancy Mace, the first female graduate of the Citadel; state Sen. Lee Bright; and businessman Richard Cash. They have criticized the incumbent for working with Democrats on immigration and other issues.

Separately, Senate GOP opposition is solidifying against Obama’s nomination of Patricia Millett to join the U.S. Court of Appeals for the District of Columbia Circuit.

Some Democrats are hinting that if Millett is defeated, they’d consider ramming through rules changes to make it harder for the Senate minority party — currently Republicans — to block nominations.