In 1847, he was consecrated Bishop of Cape Town in Westminster Abbey, along with three bishops for Australia, and arrived in his diocese, the boundaries of which were undefined, in February the following year.
Soon after arriving he set out on a journey to explore his diocese, accompanied by James Green, who was to be rector of Pietermaritzburg in the Colony of Natal. On reaching Grahamstown he ordained William Long, with whom he was to come into conflict later.
Robert Gray was married on 6 September 1836 to Sophy Myddleton, the daughter of county squire Richard Wharton Myddleton of Durham and Yorkshire, and founded St George's Grammar School, in 1848 in the shadow of the St. George's Cathedral and Diocesan College, or Bishops as it is commonly known, in Rondebosch, Cape Town, in 1849. A notable part of his life concerned his feud with Colenso about Colenso's alleged heretical opinions. [3] Gray and his wife looked after the children of Sandile kaNgqika, including his daughter Emma, aiming to provide influence over the Gaika people.[4]