He endorsed his son, John Conyers III, to replace him during an interview Tuesday morning on WPZR's "The Mildred Gaddis Show."
“My legacy can’t be compromised or diminished in any way by what we’re going through now,” Conyers said. “This too shall pass. My legacy will continue through my children.”
The announcement came amid growing calls for Conyers’s resignation.
House Minority Leader Nancy Pelosi, D-California, called for the congressman to step down last week just days after she called him an “icon” of the Democratic Party. Conyers, who was first elected to the U.S. House of Representatives in 1964, was the longest-serving member of Congress, with 52 years of service.
"The allegations against Congressman Conyers ... are serious, disappointing and very credible," Pelosi said. "It's very sad. The brave women who have come forward are owed justice. I pray for Congressman Conyers and his family, and wish them well; however, Congressman Conyers should resign."
>> Related: Rep. John Conyers hospitalized amid sex harassment accusations, calls for resignation
On Monday, a woman who said she worked for Conyers for more than a decade said he slid his hand up her skirt and rubbed her thighs while she was sitting next to him in the front row of a church.
Elisa Grubbs made the allegation in an affidavit released late Monday by her attorney, Lisa Bloom. Grubbs is the cousin of another accuser, Marion Brown, who reached a confidential settlement with the congressman over sexual harassment allegations, but broke the confidentiality agreement to speak publicly last week.
Brown, who worked for Conyers in a variety of capacities from 2003 until 2014, told NBC's the "Today" show last week that the lawmaker "violated my body, he's touched me in different ways."
“It was very uncomfortable and very unprofessional," she said. “It was sexual harassment -- violating my body, propositioning me, inviting me to hotels with the guise of discussing business and propositioning for sex.”
At least four other former staff members have accused him of inappropriate behavior, the Detroit Free Press reported. He has denied any wrongdoing.
Michigan state Sen. Ian Conyers, the 29-year-old grandson of John Conyers's brother, earlier told The New York Times that the 88-year-old Democratic congressman planned to announce that he would not run for re-election. The elder Conyers's attorney, Arnold Reed, declined to address the report Tuesday.
"I have not spoken to Ian Conyers and no one is aware of the congressman's plans except he and I and his wife," Reed wrote.
Reed said at a news conference last week that John Conyers alone would decide whether he would step down amid growing pressure from his Democratic and Republican colleagues.
"They're not going to determine whether Congressman Conyers resigns," Reed said. "He's not thinking about that. He's thinking about his health -- he's thinking about getting well."
John Conyers was hospitalized last week with what a family spokesman called a stress-related illness after complaining of feeling light-headed.
Reed has said John Conyers' health would be the paramount consideration in whether he decides to step down from his House seat.
The Associated Press contributed to this report.