Mildmay was the second son of William Mildmay, of Moulsham, Essex. He was admitted at Emmanuel College, Cambridge in 1614 and admitted at the Inner Temple in 1616. He was adopted as heir to his uncle Sir Gawen Hervey in 1622 and was directed to take the name of Hervey before Mildmay, this practice being followed thereafter by his family.[1] As a result he inherited the manor house at Marks and became Carew Mildmay of Marks where he lived during the Civil War when he commanded a Parliamentary regiment. At one stage Marks was besieged by royalist forces and he only escaped capture by swimming the moat.[2] He also purchased an interest in part of the park that had been associated with the Royal Palace at Havering when that was broken up and disposed in 1652.[3]
He was appointed Groom of his Majesty's Jewels and Plate on 5 May 1605, his relation Henry Mildmay was Master of the Jewel House.[4]