His departure follows verbal attacks from Economic Freedom Fighters leader Julius Malema
Supporters are backing Ndlozi to challenge incumbent Marshall Dlamini as the battle hots up