 Swartz was born in Kimberley, Northern Cape and educated at St Paul's Theological College, Grahamstown.He was ordained a priest in 1980.Swartz began his ordained ministry as a curate at St Matthias' Church in Welkom(1980–81) and, following his ordination to the priesthood, was appointed as the rector of St Philip’s Bloemfontein (1981 – 1987), serving for some of this time as chaplain to the Company of St Augustine (Diocesan Director of Ordinands).