Supreme Court Justice Antonin Scalia said Wednesday that the courts ultimately will have to determine the legality of surveillance programs by the National Security Agency.

And he's not sure that's a good thing in an era of complex security threats against the United States.

Scalia told the Northern Virginia Technology Council that questions about how much information the NSA can collect about Americans' telephone calls and under what circumstances the agency can monitor conversations are best answered by the elected branches of government.

But he said that the Supreme Court took that power for itself in 1960s-era expansions of privacy rights, including prohibitions on wiretapping without a judge's approval.

"The consequence of that is that whether the NSA can do the stuff it's been doing ... which used to be a question for the people ... will now be resolved by the branch of government that knows the least about the issues in question, the branch that knows the least about the extent of the threat against which the wiretapping is directed," he said.

Scalia did not raise the issue in his speech, but instead responded to a question about it. He repeatedly used the term "wiretap" in his comments, but indicated later that he was speaking more generally about NSA surveillance, including the massive collection of Americans' phone records.

In July, following the disclosures by NSA leaker Edward Snowden about the extent of the agency's surveillance programs, the Electronic Privacy Information Center asked the Supreme Court directly to bar NSA from collecting phone call records on millions of U.S. customers. The court has not yet decided whether to hear the case.

Civil liberties groups also have filed lawsuits challenging the program as a violation of Americans' privacy.

Earlier this year, the Court ruled in a 5-4 vote that clients represented by the American Civil Liberties Union lacked standing to challenge a 2008 law under which the NSA conducts aspects of its surveillance. Scalia voted with the majority to turn away that challenge to the law.

Scalia said the Constitution calls for a balancing test to determine whether any search or seizure is reasonable, and that depends on the threat that is posed — another question he said courts are ill-equipped to answer.

He talked about the pat-downs and other searches that occur at airports as an example of that balancing act.

"That's a terrible intrusion of privacy," he said. "But you're willing to do it because of the seriousness of the threat."

Among the issues in the challenges to the collection of phone records is that the court ruled in 1979 that police ordinarily do not need a judge-approved warrant to get information about who someone has called, as opposed to monitoring the call itself.

As for the question about tech companies' obligations to inform clients about an illegal intrusion of their information, Scalia said that, yes, a company should speak up if it knows a customer has had its data illegally seized. "But it's pretty hard to know that. ... If it's a governmental wiretap, presumably it's been approved by somebody, some lawyer expert in the field who said it was OK, and you better be damn sure you're right before you blow the cover."

Scalia said that before the court's 1967 opinion on wiretapping, the high court held the view that there were no constitutional prohibitions on wiretaps because conversations were not explicitly granted privacy protection under the Fourth Amendment, which protects against Americans against unreasonable search and seizure of "their persons, houses, papers, and effects."

But he said then the Warren court stepped in and found that "there's a generalized right of privacy that comes from penumbras and emanations, blah blah blah, garbage."

A bipartisan group of senators on Wednesday discussed legislation they are offering that would end the NSA's collection of Americans' phone records.

___

Associated Press writer Mark Sherman contributed to this report.