Greenhoff was born in Barnsley, and as a youngster played for Yorkshire Schoolboys. He joined Manchester United as a youth player in August 1968, and made his first-team debut against Ipswich Town on 8 September 1973. He helped Manchester United win the 1977 FA Cup, by which time he had developed a partnership with Martin Buchan in central defence. He was very versatile, starting his career as a midfielder and finishing it as a centre back of international class. Greenhoff scored 17 goals in his Manchester United career and played 271 games between 1973 and 1979. His brother, Jimmy, also played for Manchester United.
Greenhoff was brought to Manchester United as one of the last of the Busby Babes, scouted by Joe Armstrong - his childhood affinity with United, due to Barnsley born Tommy Taylor starring, played a huge role in persuading him to join the European Champions despite interest from the more local club Rotherham United, managed by Tommy Docherty. A succession of injuries prevented him from making his debut as a teenager until Docherty took over at Old Trafford. Upon greeting Greenhoff, Tommy exclaimed, "I've got you at last. The long way round, but I've got you at last." Following his debut at Portman Road, Greenhoff was essentially an ever-present, one of the most vital components of Docherty's exciting young team. Despite relegation in the 1973/74 season, Greenhoff's performances had been one of the few plusses, causing him to be named the Supporters Player of the Season.
Though used more often in midfield, Greenhoff was more comfortable playing at centre half - due to his talent on the ball and willingness to work hard for the team, he ended up being played in every position for the United first team - including in goal, when Alex Stepney was injured in a game at Birmingham City. Eventually Greenhoff was moved back into his preferred centre half position - by accident, according to Docherty, as his team were chasing a goal in a cup tie - and the modern United centre half was born. Disappointment in the 1976 FA Cup Final, where the image of a tearful Greenhoff being consoled by his manager became an iconic one, was followed by the best season of the players career in 1976-77. A man of the match performance against Ajax in the UEFA Cup came in a strong start to the season, where he also became a regular at the heart of the defence in the England national team. In the 1977 FA Cup Final, it is commonly agreed that Greenhoff was the man of the match.
Greenhoff left for Leeds United in 1979 for £350,000, which at the time was a record transfer for anyone leaving Manchester United. He was given a free transfer when Leeds were relegated in 1982. During his time with Leeds there had been concerns expressed about his weight. He returned to the game as player-coach at Rochdale after his brother had been appointed player-manager in 1983. He left the club when Jimmy was sacked in March 1984 but stayed in the area.
He won 18 caps for England, and appeared once for England B. Greenhoff played four times for the England under 23 team, scoring once and made his debut for England on 8 May 1976 in a 1-0 win at Ninian Park, Cardiff against Wales. His last game came on 31 May 1980 in a friendly against Australia at the Sydney Cricket Ground, Sydney.