 Despite continuing friction with the local Congregationalists, Bailey was able to build a church and parsonage in 1770-71.Hostilities towards Bailey continued, especially when he continued to profess Loyalty to the British Crown, and he moved with his family to Nova Scotia in 1779, after American Patriots twice made attempts on his life.He served in the parish of Cornwallis for a period and then moved to Annapolis Royal where he remained for the rest of his life.