A succession of top Republicans have begun distancing themselves fromMitt Romney after the former presidential hopeful made a series of blunt comments about minority groups as he sought to explain why he lost the race for the White House.

Romney, who less than two weeks ago was still a real contender to be America's next president, appears to have dramatically damaged his chances of becoming an influential party figure in the future.

Some moderate conservatives have started calling for Republicans to work harder to attract Hispanic voters, other minorities and women and to appear less extreme in some of its ideological stances such as tax cuts for the wealthy. Louisiana governor Bobby Jindal even went so far as to say the party had to stop being "the stupid party".

But on a conference call with his defeated donors last week Romney sparked outrage among many Republican figures after he appeared to say Obama had won support by "giving a lot of stuff" to some voters such as Hispanics, black Americans and young people in the form of healthcare, free contraceptives and forgiveness of college loan interests. "In each case, they were very generous in what they gave to those groups," Romney said.

Those comments caused Jindal to criticise the former governor of Massachusetts in a CNN interview. "I absolutely reject that notion, that description. If you want voters to like you, the first thing you need to do is like them yourself," Jindal said.

He was far from alone. New Hampshire senator Kelly Ayotte, who was once talked about as a potential Romney running mate, told MSNBC: "I don't agree with the comments. I think the campaign is over, and what the voters are looking for us to do is to accept their votes and then go forward, and we've got some big challenges that need to be resolved."

New Jersey governor Chris Christie, who caused ructions within the Romney campaign by praising President Barack Obama's handling of Hurricane Sandy, also spoke to the left-leaning cable channel and pulled no punches. "You can't expect to be a leader of all the people and [be so] divisive. Someone asked me: 'Why did Mitt Romney lose?' And I said: 'Because he got less votes than Barack Obama. That's why'," Christie said.

As the Republican party digests its defeat by Obama, a profound debate is taking place about its future direction given the breakdown of the results. Romney won the white vote with 59%, according to exit polls, but minorities coalesced around the president with 93% of blacks and 71% of Hispanics backing Obama. The changing demographics of the US mean those minority voters are only likely to get more powerful in future elections.

The attacks on Romney are coming from people associated with an emerging, modernising wing of the Republican party who want the party to broaden its appeal and believe its recent shift to the right runs the risk of making the party far too reliant on older, white voters.

However, Romney's comments during the donor phone call have surprised many experts. It was widely thought that Romney, who governed Massachusetts as a moderate Republican, was uncomfortable with some of the conservative positions he had to take to win his party's 2012 nomination. But his remarks suggest that he is still going to hew to a more conservative line that echoes some of the infamous phrases he made on a secretly-taped video at a donor meeting about the "47%" of Americans who pay no income tax.

One Republican politician, Idaho congressman Raul Labrador, was brutal in his assessment of Romney's position in the party. "He's not going to be running for anything in the future," he told the Washington Post.

That is true. Romney has no natural base in the Republican party, outside the world of high finance and big business. Unlike failed 2008 challenger John McCain – who remains very active in the Senate – Romney has no elected office to use as a platform. That is going to be make if difficult for Romney to influence either public opinion or Republican politicians.

Nor is he now trusted by many people on either wing of the Republican party. Moderates look set to disavow him as a divisive conservative who failed to adapt to a changed American electorate, while the right wing still sees him as an unconvincing convert to their causes.