In 1738, Samuel Townsend purchased the property now known as Raynham Hall. He was originally from nearby Jericho, and probably moved to Oyster Bay because of its active harbor, which would have benefited his mercantile shipping business. He co-owned the business with his brother, Jacob, who moved into the house next door. Samuel’s property consisted of a four-room frame house on a sizeable plot of land with an apple orchard across the street and a narrow meadow leading down to the harbor.


By 1765 Samuel and his brother owned four ships, which sailed to Europe, South America and the West Indies, and imported an impressive range of goods such as lumber, molasses, pottery, wine, assorted fabrics, dye and rum. In addition to their international shipping business, Samuel operated a general store from his home, providing local access to a wide variety of imported wares. He was an active member of both local and state government, as Oyster Bay’s Justice of the Peace and Town Clerk, a member of the New York Provincial Congress from 1774 to 1777, and a New York State Senator from 1786 to 1790.

Although most of Oyster Bay favored the Loyalists during the American Revolution, Samuel’s sympathies were with the Patriots. Following the colonists’ defeat in 1776 at the Battle of Long Island, British forces occupied all of New York City and Long Island until the end of the war in 1783. The Townsend family, unlike many Patriots who fled, decided to stay in their home throughout the occupation.

Hardships during the Revolutionary War

In September of 1777, British soldiers came to the door, demanding to see “Sam” Townsend. They had come not only to arrest Samuel for his Patriotic beliefs, but to imprison him on one of the notorious prison ships in New York Harbor, where horrendous conditions resulted in the deaths of over 10,000 captives by the end of the war. A British officer strode into the front hall, smashed a hunting rifle that was mounted above the mantle, and declared that a rebel had no right to possess such a weapon. He then motioned to a portrait hanging in the parlor of their oldest son, Solomon, and demanded to see him as well. When told that Solomon was at sea, he expressed regret that they could not arrest him as well. Outside the house, wife Sarah and daughters Sally and Phebe were frantic, afraid they might never see Samuel again.  A large group of neighbors gathered in the yard, expressing regret and sympathy for the Townsends, despite differing politics.

Samuel was led away with only a single change of clothes, through the village and towards Jericho, traveling up the long hill in Pine Hollow. Coming down the hill were Thomas Buchanan and his wife Almy, riding in a Phaeton carriage, with Samuel’s daughter Audrey alongside on horseback. Though he was Tory, Buchanan was very close to the Townsends. He was Samuel’s brother-in-law, was also in the shipping business, and had hired Samuel’s son Solomon to captain his merchant ship, the Glasgow.

When Buchanan saw Samuel being led away, he took Audrey’s horse and followed the soldiers to Jericho, where he paid a huge sum of money, several thousand pounds, to secure Samuel’s freedom. To the great relief of the family and friends in Oyster Bay, Samuel returned home, unharmed. Following the end of the war in 1783, when all British were required to evacuate, Thomas Buchanan’s great loyalty and friendship were remembered, and he was allowed to stay and continue his successful merchant business in New York.

In his 1846 book, “Revolutionary Incidents in Queens County”, Henry Onderdonk described another compelling account of Oyster Bay during the war, which mentions the Townsend house: “A great variety of troops lay at Oyster Bay village during the war. Delancy’s Corps was the first. Fanning’s Corps, in charge of Major Grant, lay here one summer. They were rude and ill-behaved…The streets were garnished with sentry boxes to protect the patrol from the weather. These paraded the streets after 9 o’clock at night, when no one was allowed to pass without a countersign. One evening…John Weeks, when challenged by the sentinel, instead of giving the countersign, left the road and ran off across the fields. He was seized, tried, and sentenced to be whipped. He was accordingly tied up to a locust tree in front of Townsend’s, but before he received the full measure of his punishment, the cries of the youth and the frantic appeals of his mother and sister so wrought on the people, that by their interference he was set at liberty.”

The Homestead becomes British Headquarters

For a six-month period from 1778 to 1779, the Townsend property actually served as headquarters for a regiment of 300 British troops called the Queen’s Rangers, and their commander, Lt. Col. John Graves Simcoe, who quartered himself in the house, alongside the family. Daily officers’ meetings were held in the front parlor, and the presence of British officers in the house became an everyday fact of life. During that time, Simcoe was visited several times by his close friend, British officer Major John André, who would be later be hanged as a spy for his role in helping Benedict Arnold turn traitor in 1780. After the war, Lt. Col. Simcoe founded the city of Toronto, where he served as Governor of Upper Canada.

America’s First Valentine

John Graves Simcoe also had a more personal link with the Townsend Family. Indeed, Raynham Hall is home to the first known Valentine in the United States, addressed to Sarah from Simcoe, and presented to her on February 14th, 1779. The Valentine alludes to the difficulty of loving an enemy. The poem begins:

Fairest Maid, where all is fair

Beauty’s pride and Nature’s care;

To you my heart I must resign

O choose me for your Valentine!


Click here to read the entire poem…


The Valentine was said to have been found among Sarah’s possessions after her death at the age of 82.  Sarah never married, and we may never know what effect the Valentine might have had on her life, nor the true nature of her relationship with Simcoe.

Robert Townsend and the Culper Spy Ring

At the time he was recruited into the American spy network in 1778, Robert Townsend operated a New York City-based merchant shipping firm with his brother William and cousin John. By using his work as a merchant as a cover, Robert could move about the docks of Manhattan observing British troop movements, without arousing suspicion.

Under the code name “Culper Junior,” Robert formed the first link in a chain of agents who came to be known as the Culper Spy Ring. Using a special invisible ink formula, invented by John Jay’s brother Sir James Jay, as well as an elaborate numeric code, the spies supplied Washington with critical information about New York City and Long Island. Robert forwarded messages via courier (usually Long Island tavern keeper Austin Roe) to the Culper Ring’s headquarters in Setauket, Long Island. Once there, the messages would be secreted by whaleboat to Connecticut, and eventually to George Washington’s headquarters. There Washington could apply the re-agent solution that would make the invisible message reappear.  The process of gathering and forwarding information was both slow and hazardous, and it caused continual friction between Washington and his field agents. Repeated attempts to speed the messages met with failure, and Washington’s frustration is clear from his correspondence. “It is of little avail to be told of things after they have become a matter of public notoriety,” he wrote in June of 1779.

Despite the shortcomings of the system, Robert and his comrades provided valuable service to the American cause. By gathering information on British troop movements, they alerted Washington to the possibility of attack. Likewise, they broke the news that the British were planning to undermine the war effort by flooding New York with counterfeit American currency.

The greatest coup of the Culper Spy Ring was alerting Washington to a planned British attack on the French fleet landing at Newport, Rhode Island. With this timely piece of intelligence, Washington was able to bluff the enemy into believing he would attack New York City. This forced the British to withdraw their attack force, and the French were able to disembark without hindrance.

Robert Townsend served his country well, and at great risk to himself and his family, as the punishment for spying was death by hanging. Though he moved back to Raynham Hall following his father’s death in 1790 and lived for years with his sisters Sarah and Phebe, he kept his involvement in the Culper Spy Ring a total secret from his family and friends for the remainder of his life. Indeed, Robert’s involvement in the Culper Spy Ring was not uncovered until the 1930s, when local historian Morton Pennypacker used handwriting analysis to prove the true identity of Culper Junior.

 Slavery at the Townsends

Samuel, his wife Sarah and their children were not the only occupants of their home. The Townsends, typical of wealthy New York families, owned many slaves who labored to maintain the house and grounds.


Other slaves from Oyster Bay are recorded as having come to the Townsend’s general store to purchase goods for their masters, and the “Negro ledger,” kept by sons William and David, records many purchases made during the 1760s and 1770s.

The Samuel Townsend Family

Samuel Townsend, 1717-1790

married Sarah Stoddard, 1724-1800

Solomon (died aged 4 months) 1743 – 1744

Captain Solomon, 1746-1811

From an early age Solomon was educated in the intricacies of his father’s shipping business, going to sea as a cabin boy at age fourteen and becoming a ship’s captain at twenty, first for his father’s ship, the brig Sally, and later for Buchanan’s 300 ton ship Glasgow, making many successful transatlantic voyages. During the war, with a Loyalist employer, Solomon’s own patriotic beliefs led him to give up career at sea in 1778. Landing the Glasgow in London, he traveled to Paris where he met with Benjamin Franklin, and swore his allegiance to the new United States. Returning to America, but unable to return home due to the occupation, he spent the last years of the war with his cousin Peter Townsend, owner of the Sterling Ironworks in Orange County, New York. Earlier that year Peter Townsend had forged for General Washington “The Great Chain”, an enormous iron chain which was stretched across the Hudson River at West Point to block the passage of British warships. Solomon learned the iron business, and married Peter’s daughter Anne in 1782. After the war he established the Augusta Forge nearby, as well as an anchor shop in New York City. He went into the shipping business, for a period with his brother Robert, and dealt primarily with iron and anchors. Following in his father’s footsteps, he was elected to the New York Legislature and was serving in Albany when he died in 1811 at age 65.

Samuel, 1749-1773

Samuel was married to his cousin Esther Townsend and they had a son named William Penn. Samuel was a merchant in the family merchant trade, and while on business in Wilmington, North Carolina in 1773, he caught a fever and died at the age of 24. Tragically, just two months later, his son Penn also died, aged 2 in Oyster Bay, leaving Esther a deeply grieving young widow.

Phebe (died aged 3 months) 1751-1752

Robert (Culper, Jr.), 1753-1838

See Robert Townsend and the Culper Spy Ring above.

Audrey, 1755-1829

Audrey married Captain James Farley, a sea captain who had commanded one of her father’s vessels before the Revolutionary War, and was a close associate of her brother Captain Solomon Townsend. She and James did not have children, and lived in Oyster Bay their entire lives, though Capt. Farley was often at sea.

William, 1757-1805

William was a clerk in his brother Robert’s mercantile business in New York City during the war, rooming with Robert and cousin John in an apartment above the shop described as “Bachelor’s Hall”.  After the war he returned to Oyster Bay. Considered by some family members to be the most handsome of the Townsend sons, he was called “the flower of the family”. William died unmarried in the winter of 1805 at age 48. While he was cutting a boat out of the ice in Oyster Bay harbor, he slipped, fell into the icy waters and drowned.

David, 1759-1785

Also engaged in the family business, David died unmarried of hasty consumption (tuberculosis) following the Revolutionary War, at age 26.

Sarah (Sally), 1760-1842

Sarah, who is believed to have had a flirtation with British Commander Simcoe when she was 18 years old (link to Valentine story) spent her entire life living in Raynham Hall with family, especially her sister Phebe and husband Dr. Seeley. After her father’s death in 1790 they shared the house with her brother Robert until her death in 1842 at age 82. An original windowpane from the front of the colonial house was preserved, with a scratched message from a British officer describing her in her youth as “the adorable Miss Sally Sarah Townsend”.

Phebe, 1763-1841

Phebe lived at Raynham Hall for most of her life, and created a family scandal when she got married in 1808 at age 45 to a man who was just 26. Her husband, Dr. Ebenezer Seeley, became the owner of the Townsend family property.