James Harrison was born at Camperdown, Victoria, the second child of Victorian-born parents James Samuel Harrison, farmer, and his wife Mary Eleanor, née Harlock. He was educated at Geelong College, Melbourne High School and the Royal Military College, Duntroon, where he graduated in 1932 as an artillery specialist.
He had a varied military career including as the first Commandant of the Officer Cadet School, Portsea and culminating in becoming a member of the Military Board, as quartermaster-general (1962–63) and adjutant-general (1963–66), and General Officer Commanding (GOC) of Eastern Command, Sydney (from 1966).
He was sworn in as Governor of South Australia on 4 December 1968. His term was unremarkable. Don Dunstan later concluded: "Sir James fulfilled his role as Governor quietly and in the traditional way, and left little mark on the State".
In 1969 and 1970, both he and his wife were hospitalised, in his case with a coronary occlusion. In 1971, he and his wife set off on an overseas holiday.
Sir James died suddenly on 16 September 1971 while flying to Honolulu. He was survived by his wife and two sons, and he was cremated. His wife, born Patricia Lennox, died in 2024.[2]