Davis was born in Swansea, in south Wales. His father served as a paratrooper. Influenced by his Uncle Herman's mandolin playing, Spencer Davis began learning to play harmonica and accordion at the age of six. He then attended Dynevor School, where he passed seven GCE O-level examinations. He moved to London when he was sixteen, and began working in the civil service as a clerical officer at the Post Office Savings Bank in Hammersmith and for HM Customs and Excise; but went back to his old school to study for A-levels in languages, becoming Head Boy in 1959. In 1960 he moved to Birmingham, to read German at the University of Birmingham. In music circles, Davis was later known as "Professor".
His early musical influences were skiffle, jazz, and blues, the mainstays of popular music in the early 1960s. Influential artists include Big Bill Broonzy, Huddy Ledbetter, Buddy Holly, Davey Graham, John Martyn, Alexis Korner and Long John Baldry. By the time he was sixteen Davis was hooked on the guitar and the American rhythm and blues music making its way across the Atlantic. With few opportunities to hear R&B in south Wales, Davis sought out any performance that came to town. When he heard a Dixieland band perform a skiffle version of the R&B song "John Henry", Davis formed a band called The Saints, with Bill Perks, who later changed his name to Bill Wyman.
When Davis moved to Birmingham, as a student he often performed on stage after his teaching work day was finished. While in Birmingham, he dated Christine Perfect, who later married Fleetwood Mac's John McVie. They busked and played in folk clubs with the Ian Campbell Trio. With Perfect on piano and Davis on 12 string guitar, they performed Canadian folk songs, such as "Spring Hill" and "Nova Scotia". They also interpreted W. C. Handy and Lead Belly songs.
In 1963, Davis went to a Birmingham public house to see Muff Woody, a traditional jazz band featuring Muff and Steve Winwood. Steve, only fifteen at the time, was already gaining notice for his musical abilities. Muff, five years older than his brother, was an accomplished jazz musician. Davis persuaded them to join him and drummer Pete York as the Rhythm and Blues Quartet. Davis performed on guitar, vocals and harmonica, Steve Winwood on guitar, organ and vocals, Muff Winwood on bass, and Pete York on drums. Playing mainly R&B covers, the band performed first at the Golden Eagle pub on Hill Street, but within a year they had landed a regular gig at The Marquee, and by 1964 had adopted the name The Spencer Davis Group. The group had No 1 hits in the UK with consecutive single releases in 1966 ("Keep on Running" and "Somebody Help Me"). Steve Winwood sang lead vocals on all the Spencer Davis Group's hits up to "I'm A Man" in 1967.
The Spencer Davis Group continued after Winwood left to form Traffic in April 1967. The group split in 1969. Another version of the group with Davis and York appeared in 1973 and disbanded in late 1974. Various incarnations of the band have toured in recent years, under Davis' direction.