Vavi yesterday said union leaders who served on structures such as the ANC's national executive committee (NEC) risked causing confusion among workers and compromising their independence.
"It will compromise the individual concerned because of obvious reasons. When the ANC moves in any direction all of its leaders must without exception move in that direction."
This, Vavi said, could cause conflict if Cosatu's central committee, its highest decision-making body in between congresses, contradicted the ruling party.
"If you are also the face of the other organisation, when you now turn around to reflect the new mandate, people will be confused."
Vavi was addressing journalists in Braamfontein, Johannesburg, after the opening of Cosatu's new headquarters. He was responding to Zuma, who had earlier called for the labour federation's leaders to avail themselves for election onto the ruling party's NEC.
Zuma was guest speaker at the event which was attended by leaders of Cosatu affiliates, and who are also members of the federation's central executive committee.
Zuma had earlier said: "It is the weakness for workers not to be in the ANC and influence the direction of the ANC. Because its policies and programme could be influenced by others who are part of this broad church.
"Not just in the branch region or zone or province but at a place where decisions are taken, the NEC," he said.
"We may at times differ on strategies and tactics of achieving a prosperous South Africa, but as members of the alliance our objectives remain the same.
"The ANC as a disciplined force of the left with a bias towards the poor and the working class needs to continue working with Cosatu and the SACP and to continue with the tradition of facing challenges together."
Although Cosatu has declared its support to the current ANC leadership, there are talks about whether to back their re-election at the party's elective conference in December.
At the ANC's 52nd conference in Polokwane, Vavi rejected nomination to serve on the ANC's NEC. Zuma has urged for closer cooperation between Cosatu and the ANC, citing historical ties in the fight against apartheid.