Israeli and Palestinian officials voiced skepticism Sunday that they can move toward a peace deal, as the sides inched toward what may be the first round of significant negotiations in five years.

U.S. Secretary of State John Kerry announced late last week that an agreement has been reached that establishes the basis for resuming peace talks. He cautioned that such an agreement still needs to be formalized, suggesting that gaps remain.

In his first on-camera comment Sunday, Israeli Prime Minister Benjamin Netanyahu appeared to lower expectations by saying the talks will be tough and any agreement would have to be ratified by Israelis in a national referendum.

Netanyahu pledged to insist on Israel’s security needs above all — saying his main guiding principles will be to maintain a Jewish majority in Israel and avoid a future Palestinian state in the West Bank becoming an Iranian-backed “terror state.”

A lifelong hawk, Netanyahu has been a reluctant latecomer to the idea of Palestinian statehood, and his critics say he uses the pretext of security to avoid engaging in good-faith negotiations.

“I am committed to two objectives that must guide the result — if there will be a result. And if there will be a result, it will be put to a national referendum,” he said at the start of his weekly Cabinet meeting. “It won’t be easy. But we are entering the talks with integrity, honesty, and hope that this process is handled responsibly, seriously and to the point.”

Hard-liners have floated referendum proposals before, usually as an attempt to add an additional obstacle to any efforts to cede war-won territories as part of a future peace agreement.

Palestinian officials were silent Sunday.

Palestinian President Mahmoud Abbas has not spoken about the possible resumption of negotiations since Kerry’s announcement Friday. In an attempt to restrict official Palestinian comment, Abbas’ office said only two top aides, Nabil Abu Rdeneh and Yasser Abed Rabbo would be allowed to speak to reporters. Neither was available Sunday.

Abbas had previously refused to negotiate with Israel so long as settlement construction continued in part of his hoped-for state. Netanyahu countered by saying he would only enter talks without preconditions.

The two sides are now set to hold more talks in Washington in coming days or weeks on the framework of negotiations, meaning a resumption of talks is not yet assured.

Gaps remain on three issues Palestinians say need to be settled before talks can begin — the baseline for border talks, the extent of a possible Israeli settlement slowdown and a timetable for releasing veteran Palestinian prisoners.

The Palestinians want to establish a state in the West Bank, Gaza Strip and east Jerusalem, territories Israel captured in the 1967 Mideast War. Abbas seeks a commitment from Netanyahu that Israel’s pre-1967 border will serve as a baseline for negotiations, but the Israeli leader has refused to do so. Previous rounds of negotiations were conducted on those lines.

Israel has said it will release some Palestinian prisoners as a good will gesture, but there are few other official details to emerge about the framework of the talks.

The actual talks are to produce a deal on the borders between Israel and a future Palestine, a partition of Jerusalem, the fate of refugees and security arrangements.

“The chances for a permanent solution are not high,” Interior Minister Gideon Saar told Israel Radio. “The Palestinians are not ready to make the historic decision to end the conflict between them and us.”

The Palestinians, too, were hardly optimistic.

“We are skeptical about these talks because the Israelis are not going to stop building in the settlements and because they didn’t accept the ‘67 borders,” said Tawfiq Tirawi, a member of Abbas’ Fatah Party. “What we got in return for going back to negotiations is an American pledge that the talks will be on the ‘67 borders, and historically the Americans always gave us such pledges but they never abided by these pledges.”