The barracks originally comprised a keep, adjoining stables and a parade ground completed in 1859. In 1873 a system of recruiting areas based on counties was instituted under the Cardwell Reforms and the barracks became the depot for the 32nd (Cornwall) Regiment of Foot and the 46th (South Devonshire) Regiment of Foot. Following the Childers Reforms, the 32nd and 46th Regiments amalgamated to form the Duke of Cornwall's Light Infantry with its depot in the barracks in 1881. The keep became the gatehouse and a hospital and married quarters were added at that time.
During the Second World War the barracks were an infantry training centre. In 1959 the barracks became the depot of the Somerset and Cornwall Light Infantry and continued in that role until the regiment was disbanded in 1968.
The keep is still standing and hosts Cornwall's Regimental Museum which was founded in 1925 and displays small arms, machine guns, maps, uniforms and paintings, and George Washington's Bible, captured by the 46th Regiment of Foot in 1777. Exhibits also included Harry Patch's medals and Herbert Carter's Victoria Cross.