Smith was born in Aberdeen, the third son of Robert Smith and Barbara Abercrombie. His father was the gardener to the Earl of Fife, who was then based at Mar Lodge on Royal Deeside. His father was later the head gardener on the Duke's estate at Innes House, near Elgin. Robert Smith then attended Fordyce Academy, where he was educated until 1864.
On leaving school, he moved to Glasgow to work for the publisher William Mackenzie, the publisher of the Glasgow Post Office Directory, before working as a cashier for shipping insurance brokers Hutchinson & Brown.
Smith was one of the founder members of the Queen's Park club, based in Glasgow. At the inaugural meeting of the club, held on 9 July 1867, he was listed as the club's first captain and treasurer.
In his "Scottish Football Reminiscences and Sketches" written in 1890, David Bone describes Smith thus:
not by any means an impulsive player, but took in the situation quietly; and while no man ever worked harder in the field, or did more for a club, he was not what could be called a brilliant forward. (He) did well in the (1872) international, and considerably helped the eleven to make a drawn battle of it.