The congregation was founded when Marguerite d'Youville and three of her friends formed a religious association to care for the poor. They rented a small house in Montreal on 30 October 1738, taking in a small number of destitute persons. On 3 June 1753 the society received royal sanction, which also transferred to them the rights and privileges previously granted by letters patent in 1694 to the Frères Hospitaliers de la Croix et de Saint-Joseph, known after their founder as the Frères Charon. At that time they also took over the work of the bankrupt Frères Charon at the Hôpital Général de Montréal located outside the city walls. (In the seventeenth century, a “general hospital” was an institution that took in old people, the ill, and the poor. Medical care was dispensed at the Hôtel Dieu.)
In 1755 the sisters cared those stricken during a smallpox epidemic. As the sisters were not cloistered, they could go out to visit the sick. Those assisted included the First Nations people in Oka, who were among the benefactors who later helped rebuild the Hospital after a fire in 1765.
After 1840, the order rapidly expanded, and over the next 100 years became a major provider of health care and other social services throughout Quebec, Western and Northern Canada, and the northern United States. In 1855, the Grey Nuns were called to Toledo, Ohio, to care for many suffering from cholera.
The city residents mocked the nuns by calling them "les grises" – a phrase meaning both "the grey women" and "the drunken women", in reference to the color of their attire and d'Youville's late husband, François-Magdeleine You d’Youville (1700–1730), a notorious bootlegger. Marguerite d'Youville and her colleagues adopted the particular black and beige dress of their religious institute in 1755: despite a lack of grey colour, they kept the nickname. When a Grey Nun worked as a nurse in a hospital, she usually exchanged her taupe habit for a white one. They wore a bonnet instead of a veil, as that was more practical for everyday work.
The rule given to Marguerite d'Youville and her companions by the Sulpician priest, Father Louis Normant de Faradon, P.S.S, in 1745 received episcopal sanction in 1754, when Monseigneur de Pontbriant formed the society into an official religious community. This rule forms the basis of the present constitution, which was approved by Pope Leo XIII on 30 July 1880. Besides the three vows of poverty, chastity, and obedience, the sisters pledge themselves to devote their lives to the service of suffering humanity.