There are two principal buildings. The older structure dates back to the late 1830s and is a 2+1⁄2-story, Federal style house built of brick. It measures 36 feet (11 m) by 34 feet (10 m). The exact date of construction is not known, but the date 1839 is carved into a flagstone in the front walk, and the style of construction is consistent with architecture in the late 1830s. The second building was built in 1891. It is also built of brick on brownstone foundations and measures 52 feet (16 m) by 34 feet (10 m). Between the two buildings, there were 14 rooms on the first floor, 29 rooms on the second, and three on the third floor. The back yard contains a long brick outbuilding, which is one story high and has a pitched roof.[2] The total area of the nominated property is 4.3 acres (1.7 ha).[1]
The inn was originally built as a farmhouse by Thomas Griswold Mather, who married his first cousin Jane Ann Mather in 1834. His wife died in 1853, at which point he sold the farm and the buildings to the town of Middletown. At that time, there were 35 acres (14 ha) in the farm, with an additional 16 acres (6.5 ha) in a wood lot. The town operated it as a poor farm until 1946. By that time, town farms had been superseded by other social agencies. The town sold the property, and the new owners established a restaurant.[2]