With at least four Republicans being booted from key committee slots in the House, GOP leaders and Speaker John Boehner are taking fire from their own right wing, accused of punishing those who don't toe the leadership line all the time.

"We haven't even been told officially that we were removed from the Budget Committee," said Rep. Jason Amash of Michigan, who was ranked in 2011 as the 11th most conservative member of the House.

"We had to read it in the newspapers," Amash said while sitting next to Rep. Tim Huelskamp of Kansas, who was booted off both the Budget and Agriculture committees.

"You know, I'm a farmer, a fifth generation farmer," said Huelskamp at an event held at the Heritage Foundation just a few blocks from the U.S. Capitol.

Neither man could offer up a reason why they were taken to the proverbial woodshed, other than they didn't back the leadership 100 percent.

"It's not about principles, it's about blind obedience," said Huelskamp, who is shown by one group as voting with GOP leaders 91% of the time.

"I think it's pretty outrageous, frankly," said Amash, who backed Ron Paul for President, and openly admits he is from the more Libertarian wing of the Republican Party.

"All that took place behind closed doors, which is a problem with Washington, D.C.," said Huelskamp.

Others who were surprisingly knocked off certain committees included Rep. Walter Jones of North Carolina and Rep. David Schweikert of Arizona, who both lost their seats on the House Financial Services panel.

While Jones has sparred with his own party on the wars in Iraq and Afghanistan and other issues, he's no liberal; he recently labeled the President's fiscal cliff proposal "ridiculous" and touted his "A+ rating" against illegal immigration.

(One can only imagine what Jones' father would have said in this situation - he was an old-time Democrat from North Carolina who chaired the defunct House Merchant Marine and Fisheries Committee; Jones the Father loved to sit off the House floor and tell dirty jokes and stories to reporters and staffers while chain-smoking cigarettes in his wheelchair, his jacket covered in ashes. But we digress.)

"This is not the first time I've been punished for speaking my mind," said Huelskamp, who wasn't sure what committees he will end up on in 2013.

"It's petty, it's vindicative, and if you have any conservative principles, you will be punished," the Kansas Republican concluded.

The move by GOP leaders comes at a time when opposition is already bubbling to proposals by Speaker Boehner on the fiscal cliff, as many rank-and-file Republicans don't like the idea of raising any new tax revenue in a deal with the White House.

It is also a time when Boehner may need the votes of people like Amash, Huelskamp, Jones and Schweikert - and others on the more conservative side of the party.

Stay tuned.