Smit, 35, is set to take up the job of Sharks chief executive on July 1 but he was hoping to play in one more match - the Premiership final with the English club - before returning to Durban to pursue a career in the boardroom.
Smit has been at Saracens for two seasons following the 2011 World Cup and has been an immensely influential figure at the English club during his brief stay.
"Well, it's weird to say that my 16-year career has finally come to an end today," Smit Tweeted. "I'm so grateful for every memory and to every teammate, supporter and mostly my wife for allowing me to live my dream at the expense of her career. Thank you everyone for a magical 16 years."
Smit was schooled at Pretoria Boys High but was unable to secure a contract with the Blue Bulls despite making the SA Schools team in 1996.
He signed for the Sharks after school and made his first-class debut in 1997 at loosehead prop for the Sharks against Western Province in a Currie Cup match where he came up against the feared Garry Pagel. Smit impressed coach Ian McIntosh and was retained for the following week's clash against the touring British and Irish Lions.
The callow 19-year-old held his own in a match the tourists won 42-12 and after the game Lions prop Jason Leonard presented Smit with his jersey.
In 2000 Smit returned the favour and gave Leonard his No 2 jersey after his first start at hooker for the Springboks.
Smit went on to play 111 tests for the Springboks, leading them to the 2004 and 2009 Tri-Nations titles, triumphed at the 2007 World Cup and also secured a series win over the British and Irish Lions in 2009.
Smit is a natural leader and has enjoyed success at almost every level he has captained, which started with victory at the 1999 Sanzar-UAR Under-21 tournament - a forerunner to the current Junior World Championships.
Smit holds three Springbok records - the most capped player (111 caps), the most capped hooker (96) and the most capped skipper (83) - and is also the most capped South African player in Super Rugby history with 125 appearances for the Sharks.
In all, he played 376 first-class matches, which also included 16 appearances for French club Clermont and 46 caps for Saracens.