He was born in Birmingham on 22 April 1814, the son of Thomas Pole.
Pole was apprenticed as an engineer to Charles H. Capper in Birmingham around 1828. He then went to India in 1844 as professor of engineering at Elphinstone College, Bombay, where he had organized a course of instruction for Indian students; his health obliged him to return to England in 1848. For the next ten years he worked in London under James Simpson and James Meadows Rendel, and was appointed in 1859 to the chair of civil engineering at University College, London. With official work from the government, he served on committees which considered the application of armour to ships and fortifications (1861–1864), and the comparative advantages of Whitworth and Armstrong guns (1863–1865).
Pole was secretary to the Royal Commission on Railways (1865–1867), the Duke of Richmond's Commission on London Water (1867–1869), also taking part in the subsequent proceedings for establishing a constant supply, the Royal Commission on the Disposal of London Sewage (1882–1884), and the departmental committee on the science museums at South Kensington in 1885. In 1871 he was employed by the War Office to report on the Martini-Henry rifle, and in the same year was appointed consulting engineer in London to the Japanese government, a position through which he exercised considerable influence on the development of the Japanese railway system. He was elected a fellow of the Royal Society in 1861, in recognition of some investigations on color-blindness.
Music was also one of his chief interests. At the age of twenty-two he was appointed organist of St Marks, North Audley Street in London, in open competition, the next selected candidate being Dr E. J. Hopkins (1818–1901), who subsequently was for fifty years organist of the Temple Church. He took the degree of Bachelor of Music at Oxford in 1860, proceeding to his doctors degree in 1867, and in 1879 published his Philosophy of Music. He was largely concerned in the institution of musical degrees by the University of London in 1877, and for many years acted as one of the examiners. His mathematical tastes found congenial occupation in the study of whist, and as a contemporary to Cavendish, he was an exponent of the scientific principles and history of the game. His literary work included treatises on the steam engine and on iron construction, biographical studies of famous engineers, including Robert Stephenson and Isambard Kingdom Brunel, Sir William Fairbairn and Sir William Siemens, several books on musical subjects and on whist, and many papers for reviews and scientific periodicals.