Freshman Rep. Ahn "Joseph" Cao (R-LA) incurred the wrath of the conservative base when he became the lone Republican out of 178 to vote for the Democratic health care overhaul in November. Now, after his fundraising has dipped considerably, he's having some second thoughts.

Cao said on Thursday he won't vote for the Senate or White House template for reform, claiming the abortion restrictions aren't strong enough.

"I have conveyed to the White House that at this point I cannot support the agenda that's being pushed because of the federal funding for abortion," Cao said in an interview with CNN, PoliticsDaily's Patricia Murphy and Talking Points Memo report.

Cao's retreat reflects a level of consistency with his anti-abortion stance -- given White House and Senate templates have less severe abortion restrictions than the House bill -- but it might also be explained by a simple desire for political survival.

One day before his announcement, The Associated Press published a story noting that Cao has "seen his fundraising drop by nearly 40 percent since his vote."

AP adds, "Even in a favorable political climate for Republicans, Cao's contributions have fallen sharply since he was alone among 178 GOP House members to vote for the health care bill on Nov. 7. He raised less than $250,000 during the three months surrounding the vote, compared with nearly $400,000 the previous quarter."

Facing re-election this year, Cao's opposition to the final motion is likely to win him more support and campaign contributions from conservatives, who ardently favor its demise.

One Facebook group with over 1,000 members called "Bye Bye Joseph Cao" claims in its description, "In light of 'Republican' Congressman Joseph Cao's vote for Obama's health care bill he should be stripped of his Republican credentials and removed from office."

The group is filled with message postings calling him a "traitor" for his vote and even claiming he should be impeached.

But while opposing the legislation may help Cao earn vital funding from conservatives, it could create even more problems for him in the general election.

Cao's dilemma is that he represents a strongly Democratic district, covering most of New Orleans, while relying on a conservatives to finance his election campaigns. President Barack Obama won 75 percent of the votes in his district in the 2008 presidential election.

"Republicans acknowledge that Cao will have a tough time holding the seat," AP notes.

At the time, Cao said his vote was "a decision of conscience based on the needs of the people in my district," on CNN the morning after the vote.

Cao voted in favor of Rep. Bart Stupak's last-minute amendment imposing strict restrictions on abortion funding. The Senate bill includes less stiff regulations, and the final bill will likely be closer to the Senate version than the House template.

"Unless the abortion language changes, I cannot support the president's program," Cao said this week, predicting that the "health care vote in the House will depend on the abortion language."

The Democratic Congressional Campaign Committee issued a strong rebuke to Cao's decision.

"We’re not sure if it’s the big money health insurance interests or if it’s his Washington party bosses that have gotten Representative Cao to leave his constituents out in the cold and stand against President Obama and health insurance reform," said DCCC spokesman Jesse Ferguson.

"This is a victory for national Republicans and health insurance lobbyists who have convinced Representative Cao to change sides but a sad day for the people of New Orleans."

Cao is the first Vietnamese-American to be elected to Congress.