The US investment bank said on Tuesday Pandit had stepped down as chief executive and left the company's board with "immediate effect".
Citi said Pandit will be replaced by Mike Corbat, the bank's current head of Europe, the Middle East and Africa.
Pandit said it was "the right time for someone else to take the helm at Citigroup" after the bank "emerged from the financial crisis as a strong institution".
Michael O'Neill, Citi's chairman, said the board respected Pandit's decision to leave the bank. "Since his appointment at the start of the financial crisis until the present time, Vikram has restructured and recapitalised the company, strengthened our global franchise and refocused the business," he said.
Pandit said he "could not be leaving the company in better hands". "Mike is the right person to tackle the difficult challenges ahead, with a 29-year record of achievement and leadership at this company," he said. "I will truly miss the wonderful people throughout this organization. But I know that together with Mike they will continue to build on the progress we have made."
The bank's president and chief operating office John Havens also resigned from the company. The bank said Havens "had already been planning retirement from Citi at year-end but decided, in light of Mr Pandit's resignation, to leave the Company at this time."
The departures remove a leadership team that navigated Citigroup through 2008's global credit crisis, when taxpayers rescued the bank from collapse with a $45bn (£27bn) bailout.
Shares of New York-based Citigroup, the third-largest US bank, declined 2.6% to $35.70 in early trading in New York.
If no changes are made to Pandit's compensation package, Citigroup will have paid him about $261m in the five years since he became CEO, including $165m it paid to buying his Old Lane Partners LP hedge fund in 2007 in a deal that led to his becoming CEO, according to Bloomberg.