This living history museum consists of three structures, the circa 1740 Governor Fitch IV “law office” (orange building), the circa 1826 Downtown District Schoolhouse (red building), and the circa 1835 Norwalk Town Hall. Also located on the site is a historic cemetery, originally called the Town House Hill Cemetery.

Through the efforts of Gill & Gill in landscape design and preservation, the once dilapidated and overgrown landscape of the museum was redesigned to unify the multiple structures on Mill Hill overlooking Norwalk River to the West and Village Green to the Northeast. Work included new fencing, pathways, gardens, and accessibility features. An additional focus of the design was the master planning for a future pathway linking the South facing Mill Hill overlook to the public access along the river waterfront.