The move comes just months after the east African nation embarked on a national campaign to seize land left undeveloped by investors and return it to poor farmers.
The Tanzanian government has become increasingly concerned about land speculation by investors and the conflict it creates with local residents.
Kinondoni District Commissioner Ali Hapi said the title deed for the disputed 13 hectares (33 acres) of farmland on the outskirts of Dar es Salaam, Tanzania's commercial capital, was revoked on Oct. 28.
The Ministry of Lands, Housing and Human Settlements Development had given Sumaye a 90 day deadline to develop the disputed plot of land or face losing it, he said.
Speaking at a public rally in the port city's Mabwepande area, Hapi said Sumaye had been notified of the decision and is now banned from the land.
"Security forces will be deployed to ensure that no activities are carried out in this area pending legal formalities to transfer the ownership to the respective municipal authorities," Hapi said.
However Sumaye, who was prime minister between 1995 and 2005 and defected to the opposition in last year's election, said the decision to revoke his title deed was politically motivated.
"If they thought dispossessing me of this farmland would force me to re-join the ruling party, let them forget it. I will not go back," he told reporters in Dar es Salaam.
Since his election last year, President Magufuli has taken steps to monitor and improve government control over the country's land sector in an effort to tackle inefficiencies and corruption.
In May, the government revoked ownership of more than 1,800 hectares (4,400 acres) of land in the eastern city of Morogoro to redistribute it to local people in a bid to resolve recurring conflicts between farmers and cattle herders.
Hapi said the disputed area will soon be surveyed and plots of land distributed to local residents who met the criteria to own plots of land.
"A better land use plan will be drawn up by our experts and all residents who, in one way or another, had been involved in this dispute will be allocated with plots of land," he said.
Local residents welcomed the government move, saying their families would be better protected.
"We were living in fear due to the constant eviction threats, sometimes in the middle of the night" said Abdallah Mohammed, a resident of Mabwepande.