Trinity Church stands in the village of Milton, on the north side of Milton Road just east of its junction with Shearshop Road. It is a single-story wood frame structure, with a gable roof and clapboarded exterior. It has tall Gothic-arched windows on the front and sides, and its main entrance is set in a Gothic-arched recess, framed by pilasters and a cornice. A fully pedimented gable above has a small Gothic window. The tower rising above the roof ridge has Gothic-arched louvered openings in the square belfry stage, and pinnacles at the corners beside the steeple base. The interior houses an organ that was built in 1823, making it possibly the oldest in New England.
The church congregation was established in 1795 as an offshoot of the St. Michael's Episcopal Church located in Litchfield's main village. This church was designed and built by Oliver Dickinson, with its construction beginning in 1802 and ending in 1826. Dickinson began his design in 1802, seeking to emulate the second building of the Trinity Church in Manhattan (torn down in 1839).