"If I believe health care is a right, as I do, I can no longer support an amendment that makes that right dependent on someone's zip code," he said.
Related: A day of live updates from the campaign trail
Photos: Top Democratic presidential contenders campaign in Atlanta
Related: Which Democratic candidates have raised the most in Georgia
Map: Georgia Presidential candidate visit tracker
The Hyde Amendment is a four-decade-old law that prevents government health programs, such as Medicaid, from using federal funds for abortions except in cases of rape, incest or to save the life of the woman.
Biden’s campaign said Wednesday he still supported the amendment, a stance that was sharply criticized by abortion rights supporters, party activists and other Democratic presidential candidates.
Some of the most biting criticism came from former Rep. Beto O'Rourke, who attended the same IWillVote fundraiser. He told CBS News that Biden was "absolutely wrong" for supporting the amendment.
Biden has often cited his Catholic religion for explaining his abortion stances, which include a 1981 vote for a constitutional amendment that would have allowed states to overturn the Supreme Court decision that legalized abortion.
But he said he was forced to shift his stance because “women’s rights and women’s health are under assault like we haven’t seen in the last 50 years.”
“Folks, times have changed,” Biden said. “I don’t think these guys are going to let up.”