Brown built the Second Empire-style home, which he named Lambton Lodge, between 1874 and 1876. In 1880, he died in the house after being shot in the leg by a disgruntled employee at The Globe newspaper which he founded.
Between 1889 and 1916, Duncan Coulson, president of the Bank of Toronto, lived in the house with his wife Eliza and three children. Following Coulson's death, the Canadian National Institute for the Blind obtained the house in 1920 and used it for office space until 1956. A school for the blind was attached in 1920, which was later replaced by a school for developmentally-challenged children, and demolished in 1984.
George Brown House was designated a National Historic Site of Canada in 1976, but was then in a bad state of disrepair. Threatened by demolition, the Ontario Heritage Trust intervened. The agency restored the house and re-opened it in 1989 as a conference centre with tenant offices on the upper floors. Archaeological excavations conducted in 1987 and 1988 revealed over 5,000 artifacts. These artifacts have provided insights into the construction of the house as well as the landscape surrounding it and include a collectible pint corker containing the letters “William Robertson”, a silver ring and amber bead attributed to the Coulson period, and a St. George penny token from the 1850s. The house has also been featured on the HBO series Ghost Trackers.
George Brown House was designed by William Irving and Edward Hutchings in the Second Empire style with Italianate detailing. It is a red brick house characterized by Second Empire features such as pavilion massing and a grey slate mansard roof with window dormers. The carved stone doorcase is pronounced in a way that might be more expected of an institution than a private dwelling. The 1987 and 1988 archeological excavations revealed a unique "shell wall" below ground—a double foundation. An ornamental cast iron fence and gate outlines the property along Beverley and Baldwin Streets. It rests on a red brick and stone base and compliments the façade.
The interior was organized on a Georgian centre hall plan, with the main floor containing public rooms, and the upper two floors containing private rooms. Twelve of the original fifteen fireplaces remain, and the drawing room's polished marble mantel has the initials of George and Anne Brown entwined on the cartouche. The Coulson family hired Toronto architect David Brass Dick to remodel the dining room in an Art Nouveau style in the 1890s, along with the ornate front hall fireplace.