The first Greyhounds team formed in 1919, playing in the now defunct Upper-Peninsula League. The team's coach was George MacNamara. He suggested the team be called the Greyhounds since, "a greyhound is much faster than a wolf." That reference was to the already established rival club, the Sudbury Wolves.
A couple of seasons later, the Greyhounds switched to the Northern Ontario Hockey Association Senior "A" division. The team won the Senior A championship in 1921, 1923, 1924 and 1925. The 1924 Greyhounds also won the Allan Cup, becoming the only team from Sault Ste. Marie to do so. In October 1925, the club received an offer from New York to play as the Knickerbockers in the Eastern Amateur Hockey League. The Greyhounds joined the Central Amateur Hockey Association, a division of the United States Amateur Hockey Association for the 1925–26 season. After the season, several players joined the professional ranks and the team folded.
In 1929, a junior Greyhounds team was organized, competing in the Northern Ontario Junior Hockey League. The juniors won the league championship four consecutive years from 1928 to 1931, and added a fifth title in 1942. Junior hockey in Sault Ste. Marie came to an abrupt end in 1945, when the Gouin Street Arena was destroyed by fire.
The senior Greyhounds team was revived in 1948. The new team played out of a temporary home at Pullar Stadium, in Sault Sainte Marie, Michigan, U.S.A., until the Memorial Gardens opened in 1949. The senior Greyhounds won the NOHA championship four times, in 1950, 1951, 1952 and 1955. This team folded, along with the league, after the 1958–59 season.
The current Greyhounds Junior A franchise was founded in 1962 as a member of the Northern Ontario Junior Hockey League. The team's founders were Angelo Bumbacco, Lloyd Prokop, Phil Suraci, Pat Esposito and Bill Kelly. The Greyhounds played for ten seasons in the NOJHL. They were extremely successful, never having a losing season, and winning the league championship three times. In 1972, the Greyhounds entered the Ontario Hockey Association as a Major Junior A expansion team. The original directors were joined by Frank Caputo and Frank Sarlo.