In 1834, at the age of 22, Col. Andrew Ege married Margaret Ann McKaleb, daughter of Maj. John McKaleb, one of Taneytown's successful merchants. After the deaths of Margaret's only brother in 1841 and John McKaleb's death in January 1843, Margaret and Andrew inherited his 420-acre estate in Taneytown. Once settled, Andrew and Margaret commenced construction on Antrim in 1844, enlisting the services of Baltimore builder Benjamin Forrester, and sculptor William Henry Rinehart.[4][5] The inn is named after Margaret's family's ancestral home in County Antrim, Ireland.[6]
On the lands, Ege operated a large slave plantation, raising 14 horses, 12 milk cows, and 18 other cattle, which produced 1,000 pounds of butter. He also cultivated 900 bushels of wheat, rye, corn, oats, potatoes, and hay on the acreage.[4] However, after Ege went bankrupt, the estate was purchased by a local farmer, whose descendants expanded the land holdings and continued to operate it as a farm until the 1940s. Subsequently, the mansion was only used as a summer home until Dort and Richard Mollett bought it in 1988.[7]
Following the death of his wife in 1851, Andrew remarried and relocated his family to the border between Missouri and Kansas.[4]