The son of Charles L. Graves and Alice Grey, the eldest sister of Viscount Grey of Fallodon, Graves was educated at Gresham's School, Holt, (like his predecessor Reith)and then at the Royal Military College, Sandhurst.
Graves was commissioned into the Royal Scots from Sandhurst in 1911. He served with the British Expeditionary Force in France, from August 1914, and was one of the first to be taken as a prisoner of war, on 26 August 1914, spending the rest of the Great War in a German prison camp. After the war, he served at the War Office in the Intelligence Branch of the Imperial General Staff from 1919 to 1925, when he left the British Army.
He joined the BBC as an administrator in 1926, was Assistant Director of Programmes from 1929 to 1932, then Empire Service Director, 1932–1935, Controller of Programmes, 1935–1938, Deputy Director-General, 1938–1942, and in 1942 succeeded Frederick Ogilvie (jointly with Robert Foot) as Director-General. Graves left the corporation in 1943 due to ill health, leaving Foot to be sole Director-General. Graves was also a member of the Arts Council of Great Britain and the Broadcasting Council of Scotland (now known as Audience Council Scotland).
In 1921, Graves married Irene Helen Bagnell, a daughter of H. W. J. Bagnell, of the Indian Civil Service, and they had two sons. The eldest, Michael, was an RAF Spitfire Squadron leader who took part in the battle of Malta in which he was awarded a DFC. He fought in the D Day invasion and took part in the first ever flight of operational jets. After the war he was a jet pilot for Rolls Royce and then Westland of Yeovil. In 1949 Michael was killed aged 28 while test flying a Wyvern.