 Brown came to Minnesota in 1820 when the land was Michigan Territory.When the Dakota Conflict of 1862 broke out, his family was captured but were not killed due to his wife's Dakota heritage.Joseph was away at the time and returned and was a participant in the Battle of Birch Coulee.