An 85-year-old Alpharetta man who was barred from the investment brokerage industry 45 years ago was accused of defrauding investors with the help of his 55-year-old son, according to federal investigators.

The U.S. Securities and Exchange Commission said Thomas D. Conrad Jr. and his son, Stuart P. Conrad, also of Alpharetta, defrauded investors in a group of hedge funds they managed that held $10.7 million.

Conrad suspended payouts to investors for more than four years starting in 2008, according to the SEC’s complaint.

The Conrads’ hedge fund ran into trouble when a large investment turned out to be a Ponzi scheme, the SEC said.

Thomas Conrad denied any of his small firm’s activities were fraudulent.

“That’s all B.S. They’re just whistling Dixie,” said Conrad. He said he built his firm to stay small enough under federal regulations to not require him to register with the SEC as a financial advisor.

“The SEC didn’t like that,” he said.

Conrad’s fund had to repay $2.3 million of “false profits” that the fund had received from the Ponzi scheme, Valhalla Investment Partners, the SEC said in its complaint.

But while dozens of investors clamored for their money after the fund suspended payouts, Conrad paid out roughly roughly $2 million to himself and his wife, including their annual salaries totaling $252,000, and $124,000 for down payments on aircraft, the SEC said.

Stuart Conrad got about $444,000 in redemptions from the fund, the SEC said. Another son, acting as a fund administrator, withdrew $26,500 to buy a truck, the SEC said. He later repaid it without interest, but didn’t disclose the transaction to investors.

Meanwhile, a daughter-in-law got an $18,000 loan from one fund to cover credit card bills. The loan was later bought by another fund run by Conrad, and never disclosed, the SEC said.

The SEC said the elder Conrad violated federal securities law by failing to disclose to investors that he had been barred from the investment industry in 1971.

The agency said Conrad also acted improperly by having the hedge fund pay extra fees to himself as a sub-manager of some of the investments, without disclosing the conflict of interest.

Conrad’s son, Stuart Conrad, “aided and abetted these violations,” said the SEC. The agency asked the court to impose civil penalties and to require the Conrads to repay money taken from investors.