"I made a big mistake [joining this group of refugees]. Even now, some refugees are watching me talking to you. If they see me talking to you maybe they will attack me," said a refugee who asked to remain anonymous for fear of his safety.

The man is one of the hundreds of refugees who are squatting in Cape Town's CBD. The group was removed by law enforcement officials on Sunday after a court order allowed the city to enforce its bylaws by removing the group and their structures.

The refugee group has claimed xenophobia is putting their lives at risk and they want to be moved to another country.

However, the anonymous refugee from inside the group said this is "nonsense".

"There are people here [within the refugee group] who are taking chances. Some people [in the group] have a real problem, but others are following them because they are tired of this country, not because of xenophobia but because of their own personal problems.

"I don't think there is xenophobia here. There are normal criminals. Even here in town, there are people trying to make trouble. That is not xenophobia, that is just crime," said the man.

The man believes certain people within the group are spreading misinformation within the group, and that some leaders maintain allegiances with their own countrymen.

The chairperson of the portfolio committee on home affairs, advocate Bongani Bongo, has called for the immediate resolution of the situation involving the refugees.

In a statement released on Tuesday, Bongo said: "We would like to urge refugees to comply with the laws of the country and with the bylaws of Cape Town.

"The actions of the refugees are even more concerning considering that the xenophobia they refer to is not a factor in Cape Town. South Africa welcomes refugees and asylum seekers," said Bongo.

“It is untenable that the situation continues to persist, despite numerous attempted interventions that have not yielded desirable results," said Bongo.

The refugee who spoke to TimesLIVE said he wants to leave the refugee group.

"I want to get out of this mess myself. I don’t like to be here. I want to go to another town, either Mossel Bay or Saldanha Bay. The government says I should go to Mossel Bay to see people from my home."

Although the man says there are "bad elements" within the group, he believes refugees should be treated with respect.

"People from SA must not be rude to refugees. We fled from countries because of wars. Nobody hates their home country, but we are scared of the war. We like our home but can't live there right now,” said the man.