 The church was one of the early small parishes begun by English families that settled in Riceville (now Navesink), New Jersey.Services were begun by the family of John Henry Stevens, from the Isle of Wight.One of Stevens daughters married Charles E. Milnor, a Philadelphia Quaker who was "read out of meeting" for marrying an Episcopalian.