Rome's Mayor Virginia Raggi, from the populist Five Star party, had said the city had to prioritise matters such as rubbish collection and corruption.
It means only Paris, Los Angeles and Budapest are left in the running after Boston and Hamburg also abandoned bids.
The International Olympic Committee is due to make a decision next September.
Committee chief Giovanni Malago said the decision meant Italy had been "made to look like fools" and would miss out on investment.
Mayor Raggi wants Rome to focus on improving waste collection
The Italian government had supported the bid and the broader Rome municipality, which includes many other small town councils, had indicated it would support the bid to overcome Five Star's opposition.
But Mr Malago said he would not go ahead without the main city council's backing.
Rome also applied to host the 2020 Olympics but pulled out of that race in 2012 because then prime minister Mario Monti feared Italy could not afford it.