Thompson was born in Halifax, Nova Scotia. He trained as a lawyer, and was called to the bar in 1865. Thompson was elected to the Nova Scotia House of Assembly in 1877 as a representative of the Conservative Party. He became the provincial attorney general the following year, in Simon Holmes' government, replaced Holmes as premier in 1882. However, he served for only two months before losing the 1882 general election to the Liberal Party. After losing the premiership, he accepted an appointment to the Nova Scotia Supreme Court.
In 1885, Thompson entered federal politics at the personal request of John A. Macdonald, becoming Minister of Justice. In that role he was the driving force behind the enactment of the Canadian Criminal Code. Thompson became prime minister in 1892, following the retirement of John Abbott. He was the first Roman Catholic to hold the position. On a trip to England in 1894, Thompson unexpectedly suffered a heart attack and died, aged 49. He is the second and most recent Canadian prime minister to have died in office, after John A. Macdonald.
Born in Halifax, Nova Scotia, to John Sparrow Thompson and Charlotte Pottinger, he was of Irish descent. Some sources say he was born on November 10, 1845, but others say 1844. Thompson married Annie Affleck (1842–1913) in 1870. Annie Thompson was strong-willed and had the same kind of spirit that had driven Agnes Macdonald (another prime minister's wife) to ride the cowcatcher of a Canadian Pacific Railway train through the British Columbia mountains. During their courtship, Thompson was forced to write love letters in shorthand because of his soon-to-be wife's disapproving parents. Thompson's family life was marred by tragedy. A daughter, Annie, died at 1, while youngest son David lived to be 2. Two other children died at birth, but the Thompsons had five children survive childhood.
Thompson was called to the Nova Scotia Bar in July 1865, and from 1878 to 1882, he served as Attorney General in the provincial government of Simon H. Holmes. He briefly held the office of Nova Scotia premier in 1882, but his government was defeated in that year's election. Thompson was always a reluctant politician.
After his resignation from government, Thompson was immediately appointed to the Nova Scotia Supreme Court by the Prime Minister of Canada, Sir John A. Macdonald. In this role, he was instrumental in founding the Dalhousie Law School in 1883. He taught law courses at Dalhousie in its early years.