 Fischer was born in Mannheim.From 1796 Fischer was trained by Maximilian von Verschaffelt before he moved to Vienna in 1799 to study architecture under Ferdinand von Hohenberg.His first design at the age of only 22, the Prinz-Carl-Palais in Munich, made him famous and he became a professor of architecture at the Academy of Fine Arts, Munich in 1808.