Ashdown House is associated with the "Winter Queen" Elizabeth of Bohemia, the sister of Charles I. Along with his house at Hamstead Marshall, it is said that the William, the first Earl of Craven built Ashdown for her, but she died in 1662 before construction began.
Although the architect is uncertain, it is thought that Craven commissioned Captain William Winde to build the Dutch-style mansion as a hunting lodge and refuge from the plague. The house features 8,000 square feet (740 m2) of living space, a large central staircase, reception rooms, interlinking drawing and sitting rooms, a kitchen, a dining room and eight bedrooms. The property includes two lodges, three cottages and a hundred acres of land. The house is isolated, and the view from the roof includes park-like grounds and gardens, and beyond, woods and pastures. Nearby is a large group of sarsen stones and Alfred's Castle, an Iron Age hill fort.
At least one of the woods of Ashdown Park predates the house. Glastonbury Abbey held the manor of Ashbury until the Dissolution of the Monasteries in 1539. A deer park was established for the Abbey in the south of the parish. It is bounded by an ancient embankment enclosing a rounded area characteristic of Medieval deer parks. The embankment would have been topped by a park pale, probably of cleft oak stakes. The park may equate to the Aysshen Wood that a terrier of the parish in 1519 recorded as covering 415 acres (168 ha). The former deer park is now the Upper Wood of Ashdown Park. Ashdown park is a designated Site of Special Scientific Interest (SSSI).
Although a few alterations were made to the house, the building remained largely as-built until it was requisitioned for use by the army during World War II. The occupation left it in a near derelict state. The National Trust has owned Ashdown House since 1956 when it was donated to the trust by Cornelia, Countess of Craven (wife of William Craven, 4th Earl of Craven). The house is tenanted, and has been renovated by recent lease holders. In 2010 Pete Townshend bought a 41-year lease on the property and in 2011 a structural renovation was begun. Public access is restricted to the stairs and roof, with broad views of Berkshire Downs. There is also public access to the neighbouring Ashdown Woods. Admission to the house is by guided tour on Wednesdays and Saturdays from April to October.