The area of Pompey was originally part of the territory traditionally occupied by the historic Onondaga, one of the Five Nations of the Iroquois tribes of the powerful Haudenosaunee, or Iroquois Confederacy. After the American Revolutionary War, when most of the Iroquois were forced to cede their land to the victorious United States, many of the Onondaga migrated to Canada. The British Crown awarded them land there for resettlement for their support during the war. New York State took over the former Iroquois lands and sold much of the public land for development (and speculation). It reserved part as the Central New York Military Tract. Veterans of the Revolution were awarded land grants in this tract as payments for service. The town was first settled by outsiders around 1789, as Yankees from New England and other migrants moved into western New York.
The town of Pompey was formed in 1789, but not completely organized until 1794, when Onondaga County was established. The original town was divided and sub-divided into many other towns in the region, including all of the towns of Fabius (1798), Tully, Preble, and Scott (the latter two now in Cortland County), along with parts of the towns of Spafford, Otisco (1806), LaFayette (1825), Onondaga (1794), Truxton and Cuyler (the latter two now in Cortland County).
The hamlet of Pompey developed about 10.5 miles (17 km) south of the main east-west Native American trail across the state, used for generations. European-American settlers improved the trail and developed it as the Genesee Road (1794) and then the Seneca Turnpike (1800), running through the villages of Cazenovia, Manlius and Onondaga Hollow (south of Syracuse). The segment of modern U.S. Route 20 (US 20), which connects Cazenovia and Skaneateles by way of Pompey and LaFayette, was not built until 1934.
The town of Pompey covers part of four separate public school districts: