 Yarosz was born the second of eight children on the North side of Pittsburgh, but when he was ten his family moved to Monaca, Pennsylvania, a suburb twenty-six miles away.His father died when he was only eight, putting economic pressure on him and his brothers.He quit school at seventeen to train for a boxing career to earn wages for his family.