Giffard was a son of Hugh Giffard of Boyton in Wiltshire, a royal justice, by Sibyl, a daughter and co-heiress of Walter de Cormeilles. He was born about 1235 and was the younger brother of Walter Giffard, who was to become Archbishop of York and whose successful career ensured the preferment of Godfrey. His sister Mabel was the abbess of Shaftesbury Abbey.
Giffard appears to have profited from his brother's position, and held the following positions:
Complaints were later made to the Pope at Rome about the way in which the Archbishop had given this and many other benefices to his brother – as Godfrey was it was claimed, "...only in minor orders and deficient in learning".
Giffard was still Chancellor when the monks of Worcester elected him as Bishop of Worcester about between 2 and 24 May 1268, on the translation of Bishop Nicholas of Ely to the See of Winchester. Henry III accepted his appointment, and he received the temporalities on 13 June 1268. After some little resistance, Archbishop Boniface of Savoy confirmed his election, but it was not until 23 September that he was consecrated by the archbishop at Canterbury and he was enthroned in Worcester Cathedral on Christmas Day 1268.