Calm has returned to the streets of Pretoria following a chaotic march against immigrants on Friday..
Kirsten signed a two-year contract in 2011, with an option to renew for another two years, but CSA said on Friday he had informed its board of directors he would be stepping down.
"Gary has a young family and it is understandable that he wishes to spend more time at home," said CSA president and board chairman Chris Nenzani.
"I would like to thank him for his tremendous contribution to the well-being of the Proteas over the past two years.
"It is obvious to all of us, whether we have been close to the Proteas or simply fans watching from the stands, the standard of excellence that he has brought to our national team."
Nenzani said CSA would need to move quickly to find a replacement, as the team have a packed schedule this year.
The Champions Trophy campaign plays out in England and Wales next month; the Proteas tour Sri Lanka in July and August; and Pakistan in November. They play home tours against India and Australia next season.
"The board will be following due process as far as this is concerned," Nenzani said.
"Once that process has been decided, we will call a full media briefing."
The Proteas have lost only two of 19 Test matches during Kirsten's tenure, securing series victories over Sri Lanka, New Zealand, England, Australia and Pakistan.
They have had less success in the shorter formats, although Kirsten was less hands on with the T20 team this season, with Russell Domingo taking over the reins.
In the past two years, South Africa has won 13 of 24 one-day internationals and half of their 18 T20 games. They were eliminated in the second round of group-stage matches at last year's World T20 tournament in Sri Lanka.
"Gary certainly leaves the Proteas in good health and, although we are disappointed to lose him as head coach, he has put the foundations in place to carry us forward," Nenzani said.
"We will certainly endeavour to retain his services in some capacity, as his experience as player, coach and mentor is not something we want to lose in its entirety."
Kirsten, a former top-order batsman for South Africa, guided India to the 2011 one-day World Cup title, and Nenzani hoped he could carry the Proteas to victory at the Champions Trophy tournament, which would end a 15-year trophy drought for the national side.
"It will be wonderful if he can complete his tenure by bringing home the ICC Champions Trophy next month," Nenzani said.
Kirsten thanked CSA for giving him the responsibility of growing the team, and for allowing him the flexibility to combine his coaching job with his family life.
"There have been many highlights over the past two years which have been well documented," Kirsten said.
"I have every confidence that the Proteas will continue to go from strength to strength."