Loveday was born near London on 19 December 1818. He was a lance corporal in the Royal Sappers and Marines, serving in Ireland and subsequently South Australia having arrived on the Royal George in 1847. He later moved into the civil service as a surveyor serving under Surveyor-General Sir Arthur Henry Freeling, and is known for mapping part of Lake Alexandrina.
Loveday married Bridget Shea in Limerick, Ireland in 1841. They had four children, Ann, Richard, Thomas and Mary. Bridget died in Adelaide in 1852.
Loveday subsequently married Susannah Sarah Sadgrove in 1852. They had ten children: Alice, Henry William, George, Henry George, Alexander William, Emily Agnes, Richard John, Frances Jane, Ernest Alfred and Edith Marion Blanch.
Loveday died in Adelaide on 15 December 1883.
Loveday Bay at the south-eastern end of Lake Alexandrina, Loveday Street in Goolwa, and Loveday Creek in the Northern Territory are named after Loveday. The settlement of Loveday in South Australia is named after one of Loveday's sons, Ernest Alfred Loveday.