Holmes was born in Woodstock, Connecticut. He was the son of David Holmes and Temperance Bishop. He graduated from Yale College in 1783. In 1784, while ministering in South Carolina, he was recruited to be the minister at the Congregational Church in Midway, Georgia. He returned to New England to be ordained in 1785 and once for health reasons between 1786 and 1787, but returned to Midway and remained there until 1791. Holmes married Mary Stiles, the daughter of Ezra Stiles, the president of Yale. Mr. Stiles was the subject of a laudatory biography penned by Holmes.