He was an England schoolboy international before joining his local club Sunderland. He was 19 when he made his first team debut in a 1-1 draw at home to Portsmouth on 4 October 1952 and for the next eleven years he was a regular in the Sunderland side. In 1955 and 1956 Sunderland twice made it through to the semi-finals of the FA Cup, but lost both games and in 1958 the club were relegated for the first time ever. Anderson was captain of a young side in the Second Division.
The strong-tackling wing-half appeared in his first England under-23 side in September 1955, but although he was to secure more under-23 honours and make an England 'B' appearance he did not win his first full cap until the 1961-62 season. After debuting against Austria he played in a 2-0 defeat away to Scotland before being selected as a squad member for the 1962 FIFA World Cup in Chile, where he was unused throughout the tournament.
Anderson seemed set to finish his career with Sunderland, but after 400 appearances and 12 years he signed for Newcastle United for £35,000 in November 1963. He assisted the Magpies in 1964-65 as they captured the Second Division title. When he was sold to Middlesbrough in 1965 he became the first, and to date only, player to captain all three of the big clubs in the north east.
He succeeded Raich Carter as Middlesbrough manager in April 1966 and remained at the club until resigning in April 1973 to be replaced by Jack Charlton. In his time the club were relegated from and promoted to the Football League second division. After leaving Middlesbrough he managed in Greece for AEK Athens FC and in England, where he became boss at Queens Park Rangers, Doncaster Rovers and Bolton Wanderers before giving up management after resigning in 1981. He continued as a scout for various clubs including Newcastle.