"Nxamalala [Zuma] has increased grants, but there are people who are stealing them by voting for opposition parties," he was quoted as saying.
"If you are in the opposition, you are like a person who comes to my house, eats my food and then insults me."
Radebe, who was previously the province's social development MEC, warned those who wanted to vote for other parties to "stay away from the grant".
His comments were made in Greytown on Tuesday in the presence of President Jacob Zuma.
Democratic Alliance provincial leader Sizwe Mchunu told the newspaper the comments were "insulting and misleading".
"The money for social grants comes from the taxes that people pay. This is purely ANC propaganda," he was quoted as saying.