William David Ormsby-Gore was born on 20 May 1918 in Westminster, London, the second son of William Ormsby-Gore, 4th Baron Harlech, a Conservative politician, and Lady Beatrice Edith Mildred Gascoyne-Cecil. His maternal great-grandfather was British Prime Minister Robert Gascoyne-Cecil, 3rd Marquess of Salisbury. He was educated at St Cyprian's School, Eton College and New College, Oxford.
A well-known story told of him at Eton is that, when a boy in his house killed himself, the housemaster called the boys together and asked if any of them had any idea why this should have happened. Ormsby-Gore put up his hand and asked, “Please sir, could it have been the food?”
In 1939 he was commissioned into the Royal Artillery (Berkshire Yeomanry Field Regiment), served in the 'Phantom' reconnaissance unit, and worked with airborne and other special units. By the end of the War he held the rank of major on the general staff.
After the war his father handed over to him all his land, and Ormsby-Gore farmed the 400 acres (1.6 km²) of the Woodhill Estate, Oswestry, Shropshire. In 1948 he was commissioned a Major in the Shropshire Yeomanry, but left in 1950.
At the 1950 general election, he was elected Member of Parliament for Oswestry, which he remained until 1961. Under Prime Minister Anthony Eden he served briefly, from November 1956 to January 1957, as Parliamentary Under-Secretary of State for Foreign Affairs; and under Prime Minister Harold Macmillan he was from 1957 to 1961 Minister of State for Foreign Affairs. After the election of U.S. President John F. Kennedy he was appointed British Ambassador to the United States on 18 October 1961. This meant that he had to take the Chiltern Hundreds on 1 June, so that he could resign from the House.