The White House said National Security Adviser Susan Rice told Afghan President Hamid Karzai that the U.S. will plan to pull all troops out of his country after 2014 if he doesn’t promptly sign a security agreement.

Karzai’s office said in a statement he told Rice he won’t back down from his refusal to sign the pact. Karzai insists the winner of an April 5 election to succeed him should sign it.

Karzai’s surprise decision to ignore Sunday’s recommendation to sign by an Afghan assembly of dignitaries has cast doubt on the future presence of American and allied troops in Afghanistan. Washington and many Afghans had hoped the pact could be signed to allow troops to stay after 2014 to train soldiers and police to face a persistent Taliban insurgency.

The mercurial Karzai also asked for further assurances from the United States that its forces will not raid Afghan homes and that America express a sincere commitment to help start stalled peace talks with the Taliban. He also reiterated his demand that the United States commit to holding free and transparent elections April 5.

“During this meeting both sides spoke about the Bilateral Security Agreement … President Karzai, besides emphasizing his previous position, also mentioned the stopping of all foreign forces operations in residential houses, the start of an honest cooperation on the peace process, and also the holding transparent and fair elections that will be on time,” the statement from his office said.

The statement further added that Karzai asked Rice to convey his concerns about the raids and peace talks to President Barack Obama so that he can “give assurances regarding the issues to the Afghan people.”

Obama has already addressed the issue of raids in a letter to Karzai last week that was read to the assembly.

In it, Obama assured Karzai that under the agreement, the U.S. will continue to respect “Afghan sovereignty.” He also said the U.S. military will not conduct raids on homes except under “extraordinary circumstances” involving urgent risks to U.S. nationals.

The U.S. has repeatedly urged Karzai to sign a deal that would allow about 8,000 American troops to stay in the country beyond a 2014 withdrawal deadline.

On Sunday, a 2,500 tribal elders and regional leaders known as a loya jirga not only overwhelmingly approved the deal after a four-day meeting but urged him to sign it by Dec. 31.

As the U.S. national security adviser met Karzai in Kabul, the White House again urged him to sign the deal before the end of the year.

White House spokesman Josh Earnest told reporters traveling along with Obama aboard Air Force One to San Francisco that America’s position was clear.

“Our view on this has been pretty clear that in order to plan for post-2014 military presence with our allies, we need to get this agreement signed and we need to get this agreement signed before the end of the year,” he said.

“Reaching this agreement was a painstaking path,” Earnest said. “We should get the deal signed and set about the work of planning the presence of the United States and our allies after the war ends next year.”

The two-term Afghan leader has insisted that the winner of an April 5 election to succeed him should be the one to sign the deal. More than $8 billion in annual funds for Afghanistan’s fledgling security forces and development assistance also are at stake.

Karzai may be concerned about his legacy, worried he might be seen as responsible for an agreement that some Afghans will likely see as selling out to foreign interests.