New York Times columnist Paul Krugman says that the Republican Party has adopted extreme anti-immigrant positions to appeal to their base, "which is, by and large, elderly white people arguing with empty chairs."

During a Sunday panel segment on ABC News, Krugman pointed out that Clint Eastwood's bizarre conversation with an empty chair at the Republican National Convention last month was illustrative of the party's base.

"Arizona is a third Hispanic," conservative columnist George Will noted. "The Republican Party spent 20 debates in the primary competing to see who could build the longest, thickest, tallest, most lethally-electrified fence. And Hispanics said, 'I detect some hostility here.' And it's going to take a long time to undo that."

Krugman agreed that the GOP's move to the extreme right during the primary had hurt their standing with minority voters.

"The Republican Party is where it is because that's where the base is," Krugman agreed. "You watch that whole primary process, Republican candidates had to appeal to their base, which is, by and large, elderly white people arguing with empty chairs."

Tea party favorite Sen. Rand Paul (R-KY) also lamented that the Republican Party had completely given up on winning certain parts of the country.

"So what I keep telling them is, maybe we need some libertarian-type Republicans who might be popular in those areas," he explained. "Maybe a less aggressive, more socially tolerant, but still fiscally conservative policy that that may be more libertarian might do better in California, might do better in Oregon, Washington, New England."

"Our problem in the presidential election is we've given up 150 electoral votes before we get started."

Watch this video from ABC's This Week, broadcast Sept. 9, 2012.