 At that time it probably looked like a fortified castle forming a part of the fortifications of the city of Brussels.It was the home of the Dukes of Brabant who also resided in the nearby city of Leuven and in the Castle of Tervuren.In the following centuries it was rebuilt, extended and improved in line with the increased prestige of the Dukes of Brabant and their successors; the Dukes of Burgundy, the Emperor Charles V, the Archduke Albert of Austria and Infanta Isabel of Spain and successive Governors of the Habsburg Netherlands.