Davers was the second surviving son of Sir Jermyn Davers, 4th Baronet, MP and Margaretta Green. He was brought up at Rushbrooke Hall in Suffolk and educated at King Edward VI School (Bury St Edmunds) and Trinity College, Cambridge, where he graduated in 1755. He then undertook the Grand Tour.
Davers became an officer in the British Army in 1758, being commissioned into the 44th (East Essex) Regiment of Foot. He served in North America during the Seven Years' War. In January 1761 he was promoted to the rank of Captain while in the service of the 99th Regiment of Foot. He was garrisoned in Ireland in 1766 and was promoted to Major.
In 1763 Davers inherited his brother's Baronetcy and estates. In the 1768 general election he was elected as the Member of Parliament for Weymouth. At the 1774 general election he was returned as MP for Bury St Edmunds. Davers held his seat in the House of Commons due to his close personal alliance with Augustus FitzRoy, 3rd Duke of Grafton. Grafton encouraged him to cultivate his family interest in Bury at the expense of Davers' brother-in-law, Frederick Hervey, 4th Earl of Bristol. Davers publicly aligned himself against William Pitt the Younger, but did not become a Whig and retained his independence.
Davers lived at Rushbrooke Hall with Frances Treice, by whom he had five illegitimate sons and three illegitimate daughters. He was rumoured to have earlier married Miss Coutts, a planter’s daughter, in America while serving in the army, and to have had a son, Rushbrook.