There are circumstantial indications (Pickford, 2003) that Grosmont may have originated as an Iron Age camp. Grosmont Castle is a major feature of the village and was the birthplace of Henry, 1st Duke of Lancaster. Grosmont was once an important medieval township. It was granted a borough charter, possibly in 1219, and by 1250 there may have been as many as 160 burgage plots. It retained its corporation status until 1857, at which time it still had a mayor and an official ale taster. The town hall replaced a former timber structure and was built in 1832 by the then landowner the 6th Duke of Beaufort, whose descendant offered it to the Grosmont Parish Council in May 1902. A market used to be held twice a week.
The Church of St Nicholas probably has ancient origins but the tower and other parts were built by Prince Edmund (son of Henry III of England and later Edmund Crouchback, 1st Earl of Lancaster) for his mother Eleanor of Provence (Queen Eleanor).The 14th century church is dedicated to St Nicholas and was restored by J. P. Seddon in 1869. It has an unusual eight-sided tower.
Grosmont Castle, along with the nearby White Castle and Skenfrith Castle, have given rise to the Three Castles Walk which links the castles and along with the Monnow Valley Walk brings visitors to the village. Grosmont is dominated by the nearby Graig Syfyrddin (or Edmunds Tump, possibly after Edmund Crouchback, 1st Earl of Lancaster).
Grosmont is linked to the Welsh Prince Owain Glyndŵr and during the Glyndwr rebellion it was the site of a battle in 1405. Glyndwr's ally and trusted Captain Rhys Gethin raised a force of maybe 8,000 men that marched on Grosmont burning the town to the ground. At this time Grosmont was a large and important settlement - only Abergavenny and Carmarthen were larger in the whole of South Wales. Prince Henry, later to become King Henry V, dispatched a force comprising men led by John Talbot, 1st Earl of Shrewsbury, Sir William Newport and Sir John Greynder from Hereford to intercept the Welsh force. They fell on them and defeated the Welsh, killing 800 to 1,000 men and capturing Owen ap Gruffydd ap Rhisiant, Glyndwr's Secretary and John Hanmer, Glyndwr's brother in law, who both survived the battle but were imprisoned in the Tower of London.
Grosmont never recovered from the attack however. Prince Henry informed his father, King Henry IV, in a letter that maybe 100 homes had been burned.
In the summer of 2006 the pub and village were the location for the film The Baker released in 2007. The Angel public house in Grosmont was owned and run by a group of villagers until December 2014 when it was taken over by Mr and Mrs Woolnough.