Born in West Ashton in Wiltshire, he was the oldest son of Richard Godolphin Long and his wife Florentina, daughter of Sir Bourchier Wrey, 6th Baronet. Long was educated at Winchester College and then went to Christ Church, Oxford, where he graduated with a Master of Arts in 1812.
Long served as major in the Royal Wiltshire Yeomanry. He entered the British House of Commons in 1835, sitting as a member of parliament (MP) for North Wiltshire for thirty years until 1865. Long was appointed Deputy Lieutenant for Somerset, for Montgomeryshire and for Wiltshire, representing the latter county also as Justice of the Peace.
On 2 August 1819, he married firstly Mary Anne, second daughter of the politician and lawyer Archibald Colquhoun in Easter Kilpatrick in Dunbartonshire and had by her six children, three daughters and three sons. After her death in 1856, Long remarried Mary Bickerton, oldest daughter of Rear-Admiral Sir James Hillyar and widow of Sir Cecil Bisshopp, 10th Baronet in St George's, Hanover Square in London on 15 April 1857. By her he had another son. Long died after short illness, aged 73 at Torquay in Devon and was buried at his birthplace, near his home Rood Ashton House. He was survived by his second wife until 1891.
His second son was the politician Richard Penruddocke Long, who succeeded his father in Parliament and his youngest son was the soldier Lieutenant Walter Hillyar Colquhoun Long. The latter was involved in the siege at Lydenburg, South Africa, during the First Boer War. Criticised for his handling of the siege, he was later court-martialled and subsequently jumped to his death from a 4th floor window of the Grosvenor Hotel in London.