Supreme Court Justice Antonin Scalia says the courts will ultimately have to determine the legality of wiretapping by the National Security Agency.

And he’s not sure that’s a good thing.

Scalia addressed NSA wiretapping Wednesday in a speech to the Northern Virginia Technology Council when he was asked about technology companies’ roles in protecting customers’ privacy when their data has been unconstitutionally collected.

Scalia said the high court originally ruled that there were no constitutional prohibitions on wiretaps because conversations were not explicitly granted privacy protection under the Fourth Amendment, which protects against unreasonable search and seizure of “their persons, houses, papers, and effects.”

That 1928 opinion, in Olmstead v. U.S., was overturned nearly 40 years later by the Warren court, which found, Scalia said, “there’s a generalized right of privacy that comes from penumbras and emanations, blah blah blah, garbage.”

“The consequence of that is that whether the NSA can do the stuff it’s been doing … which used to be a question for the people … will now be resolved by the branch of government that knows the least about the issues in question, the branch that knows the least about the extent of the threat against which the wiretapping is directed,” he said.

He also said the Constitution calls for a balancing test to determine whether any search or seizure is reasonable, and that depends on the threat that is posed — another question he said courts are ill-equipped to answer.

He talked about the pat-downs and other searches that occur at airports as an example of that balancing act.

“That’s a terrible intrusion of privacy,” he said. “But you’re willing to do it because of the seriousness of the threat.”

As for the question about tech companies’ obligations to inform clients about an illegal intrusion of their information, Scalia said that, yes, a company should speak up if it knows a customer has had its data illegally seized. “But it’s pretty hard to know that. … If it’s a governmental wiretap, presumably it’s been approved by somebody, some lawyer expert in the field who said it was OK, and you better be … sure you’re right before you blow the cover.”

In July, following the disclosures by NSA leaker Edward Snowden about the extent of the agency’s surveillance programs, the Electronic Privacy Information Center filed a direct appeal to the Supreme Court asking it to bar NSA from collecting phone call records on millions of U.S. customers. The court has not yet decided whether to hear the case.

Earlier this year, the court ruled in a 5-4 vote that clients represented by the American Civil Liberties Union lacked standing to challenge a 2008 law under which the NSA conducts aspects of its surveillance. Scalia voted with the majority to turn away that challenge to the law.