Baker was born in Middlesex, England, to Ezekiel John Baker and Elizabeth, née Burgess. He trained in his father's business of manufacturing firearms and also studied mineralogy and mining.[2]
He emigrated to New South Wales in 1853 as mineralogist to a mining company.[3] He was elected to the Legislative Assembly for Goldfields South at the 1869-70 election,[4] and was also a member of the Goldfields Committee. In June 1870 he was appointed to a Royal Commission into the goldfields and water supply.[5] His seat was declared vacant in November 1870 as he had accepted an office of profit under the crown.[6] He comfortably regained it at the by-election in December 1870.[7] He retained the seat at the elections in 1872,[8] and 1875.[9]
When Thomas Garrett, the Secretary for Lands in the third Robertson Ministry, resigned his office, in February 1877, Baker was appointed to succeed him, but the Government only held office till March.[1] He was appointed Secretary for Mines in the fourth Robertson ministry, in August 1877, exchanging that post for Secretary for Lands in November, and going out of office with his colleagues in December.[1] Baker was again Minister of Mines in the third Parkes ministry from December 1878 to August 1881.[1] The Goldfields districts were abolished as a result of the 1880 redistribution,[10] and Baker was the first elected of two members for the expanded district of Carcoar.[11]
In August 1881 Baker resigned as Secretary for Mines following allegations concerning his conduct as a trustee of the Milburn Creek Copper Mining Co Ltd. Julian Salomons was appointed a royal commissioner to inquire into inquire into the expenditure and distribution of £17,100, paid by the Government, under the authority of a Parliamentary vote, to the company. Salomons reported that "there was an appropriation by the trustees to themselves ... under circumstances of concealment and false statement" and that there was an inference that one of the trustees, George Waddell, had bribed a member of the Legislative Assembly, Thomas Garrett, to vote in favour of the payment.[12] Following publication of the report, on 8 November Parkes moved that Baker was guilty "of conduct unworthy of a member of this House, and seriously reflecting upon the honour and dignity of Parliament", a motion that was carried by 71 votes to 2 and the assembly then voted to expel him.[2][13][14] Two days later, Parkes moved a similar motion in relation to Garrett, which was defeated by 40 votes to 38.[15]
Baker was a candidate at the resulting by-election in December 1881, but was defeated.[16] Charges against him were subsequently dropped and Baker petitioned parliament to rescind its censure of him, which was agreed in May 1884.[2] He was again elected as the member for Carcoar at the November 1884 by-election,[17] retaining the seat at the 1885 election,[18] retiring at the election in January 1887.[1]