MILWAUKEE (Reuters) - The Presbyterian Church (USA) ordained its first openly gay minister on Saturday in Madison, Wisconsin, three months after lifting its long-standing ban on homosexual clergy.

Scott Anderson, 56, was ordained minister at the Covenant Presbyterian Church in front of a jubilant congregation.

"It was a big day for me and also for the Presbyterian Church," Anderson said in a telephone interview.

"It was very healing for me and for many people there. It was very celebratory. Presbyterians don't cheer very often, but there was some cheering," Anderson said.

Anderson, who years ago had been rejected by members of his congregation at the Bethany Presbyterian Church in Sacramento, California, left the ministry in 1990. He has been with his partner, Ian MacAllister, for 20 years.

"I never thought the day would come in my life time when the Presbyterian Church would change its mind about ordaining gays and lesbians," he said.

Since leaving the ministry, Anderson had remained active in the church. He began the process of returning to the ministry five years ago. The Presbyterian Church in July changed its constitution to allow openly gay and lesbian ministers.

"It just so happened that I was the first in line, I didn't intend to be the first," Anderson said. "What a great privilege and honor and humbling to be the first."

Anderson said he believes his ordination reflects a cultural shift throughout organized religion and public policy in the United States.

"Institutional religion has not always been a helpful conversation partner with the wider culture and my hope and prayer is that, with this action of the Presbyterian Church, we will increasingly become a more constructive player in the wider cultural discussion about gays and lesbians," he said.

(Editing by David Bailey and Peter Bohan)