Fox News host grills Tony Perkins: 'What's the damage to you' if your neighbors are gay married?

At one time, Family Research Council President Tony Perkins may have viewed Fox News as a friendly venue for his campaign against marriage equality. But on Sunday, he found himself in the minority even on the conservative network.

Following the Supreme Court's decision last week not to hear appeals against rulings that struck down same-sex marriage bans in five states, Fox News Sunday invited Perkins and conservative attorney Ted Olson to debate the issue.

"Marriage is not to affirm adults, it's for the protection of children," Perkins argued. "And if love is the only factor, where do you draw the boundaries?"

Olson, who argued in favor of marriage equality before the Supreme Court last year, noted that the court had said that the thousands of children in same-sex households also mattered.

"They deserve the right to equality, and the same respect and decency that other people have that are living right next door," Olson noted.

"Well, we know from the social science that children do best with a mom and a dad," Perkins said. "If love is the factor, what boundaries are there?"

"Do you want the sky to fall because because two people that are living next door to you?" Olson asked. "Court after court has said that allowing people of the same sex to marry the person that they love and be a part of our community, and to be treated equally does no damage to heterosexual marriage. And what court after court after court has said [is] that children living in a same-sex relationship do as well or better that people in other communities."

After Perkins went back to his "boundaries" talking point that marriage equality was a slippery slope, Fox News host Chris Wallace demanded to know what he was trying to suggest.

"What are you suggesting?" Wallace asked. "That there's going to be polygamy, that people are going to be marrying their pets?"

"I didn't say that," Perkins replied defensively.

"Alright, you and your wife live happily in this house," Wallace said. "There's a same-sex couple living here. What's the damage to you?"

"Let's talk about the wedding vendors that have been put out of business," Perkins said.

"I'm not talking about that," Wallace interrupted. "That's a different issue. I'm asking you, what's the impact on you and your family to have these people living next to you."

Perkins insisted that his children would be "taught values and morals against what I teach as a parent at home."

Olson pointed out that there was no evidence that heterosexual couples were getting divorced because LGBT people had the right to marry.

"The same argument that Mr. Perkins is making was made with respect to interracial marriages in 1967," he added. "Thirty-something states at one point prohibited interracial marriages. And talk about the color of the skin, people were making the same arguments, 'Marriage is wrong between people of different races, we have to stop that.'"

In the end, Perkins compared same-sex marriage to the Supreme Court recognizing abortion rights for women, and he predicted that it would be an issue in elections for decades to come "if the court does not allow the states and the people to deal with it."

Watch the video below from Fox News' Fox News Sunday, broadcast Oct. 12, 2014.