Carroll provisionally signed for Celtic on 22 September 1957 from junior club Irvine Meadow, however he was left with Irvine Meadow for season 1958–59, during which he scored 75 goals resulting in him being voted Junior Player of the Year in 1959. On 15 May 1959, Bobby received his trophy at the George Cinema in Bank Street from the Provost George Donaldson and won the Scottish Junior Cup on 16 May 1959. He was subsequently called into the Celtic first team as one of the so-called 'Kelly's Kids', after the then Celtic Chairman Bob Kelly, where he made is debut in a 2-1 League Cup defeat at home against Partick Thistle on 12 August 1959.
On 26 September 1962, Carroll was part of the Celtic team that travelled to Mestalla Stadium for the first leg of an Inter-Cities Fairs Cup tie against Valencia (this competition being a predecessor of the current UEFA Europa League). Valencia were holders of the cup and won the tie 4-2, with Carroll scoring both of the Celtic goals, making him the first Celtic player to score a goal in European competition.
Carroll signed for St Mirren in 1963 remaining there until 1965. In the league he made 55 appearances and scored 19 goals. He then joined Dundee United, scoring on his debut against Celtic against Celtic on 14 August 1965. After 6 appearances he soon dropped out of first team contention but spent two seasons at the club, playing over 50 times for the reserves. He was released in April 1967. He joined Coleraine in Northern Ireland, then later in season 1967-68 also played for Queen of the South. In his 12 league appearanes for Queens he scored 6 goals. He then left senior football by returning to Irvine Meadow.