 Duffy won his first professional fight by knockout over Skin Doherty around February 1, 1884 at the age of 19.In 1884, he fought three bouts with Paddy Sullivan in Massachusetts with the first two ending in draws, and the third on June 28, 1884 ending in a six round win by points decision in Gloucester according to the "Boston Daily Globe".The bout was fast and hotly contested but the referee ruled in Duffy's favor.