Garretson was married twice, first to Marion Scott Becker with whom he had three sons: Frank, Herman Jr. and Ronald. His oldest son, Frank E. Garretson, served in the Marine Corps during World War II and was decorated with the Navy Cross during the combats in Pacific. He remained in the Marines and retired as Brigadier general. Herman's second wife was Helen Huston.[3]