While many have directed anger towards the four U.S. marines urinating on dead Taliban soldiers, Rick Perry seemed to place his frustration on the issue more towards the White House.

On CNN's State of the Union Sunday morning, the Texas governor was asked his thoughts about the controversial video. Although he criticized the four marines' actions, Perry felt the potential punishment they face from the Obama administration is more alarming.

"What's really disturbing to me is this kind of over the top rhetoric from this administration and their distain for the military it appears," he said. "These kids made a mistake. There's no doubt about it. Shouldn't have done it. It's bad. But to call it a criminal act is over the top."

"The idea that this administration would go after these young people for a criminal act is again, I think it is over the top. Did they make a mistake? Absolutely. Should they be reprimanded and appropriate punish? Yes. But going after them as a criminal act? Really bad message."

WATCH: Video from CNN, which was broadcast on January 15, 2012. (Highlighted clip starts at 1:55).