A fortification was built near the present site of Koromo Castle during the Kamakura period, and the area was contested in the Sengoku period between the Imagawa clan and the Oda clan. After the establishment of the Tokugawa shogunate, the Miyake clan (formerly of Tahara) were allowed to return to Mikawa and were assigned a 10,000 koku domain on the banks of the Yasaku River. In 1600, Miyake Yasusada built a jin'ya fortified residence approximately a kilometer away from the site of the original fortification, and planted sakura trees all around it. The residence was nicknamed "Sakura-jō " (桜城).
The Miyake moved the seat of their domain to Kameyama Domain in Ise Province in 1619, but returned to Koromo from 1636–1674. After 1674, they were finally allowed to return to their ancestral Tahara Domain. Koromo then became tenryō territory ruled directly by the Tokugawa shogunate through appointed daikan administrators to 1688.
From 1836–1838, the domain suffered from crop failures and famine, which led to a number of large-scale peasant's uprisings. In 1854, the town was largely destroyed by the 1854 Ansei earthquake, and entered into the Bakumatsu period impoverished and with a minor as daimyō. The Ee ja nai ka movement found strong support in the domain. Although the domain dispatched some troops to guard the Tōkaidō and Sunpu Castle during the Boshin War, these troops surrendered to the new Meiji government without resistance.
The domain had a population of 19,253 people in 5078 households per an 1869 census. The domain maintained its primary residence (kamiyashiki) in Edo at Hanzonmon until the An'ei period (1772–1780), when it was relocated to Mita.[1]
As with most domains, Koromo Domain consisted of several discontinuous territories calculated to provide the assigned kokudaka, based on periodic cadastral surveys and projected agricultural yields.[2][3] At the end of the Edo period, Koromo Domain consisted of 25 villages in Kamo District and 6 villages neighboring Shitara District in Mikawa, and 12 villages in Kumehokujo District, Mimasaka Province, 8 villages in Shūchi District, and Haibara District, both in Tōtōmi Province.