Schmidt was born in Pozsony (known in German as Pressburg), in the Hungarian part of the Austro-Hungarian Empire (the city is now Bratislava, capital of Slovakia). His father was half Hungarian and his mother entirely Hungarian. He was a Roman Catholic.
His earliest teacher was his mother, Mária Ravasz, an accomplished pianist, who gave him a systematic instruction in the keyboard works of J. S. Bach. He received a thorough foundation in theory from Brother Felizian Moczik, the outstanding organist at the Franciscan church in Pressburg. He studied piano briefly with Theodor Leschetizky, with whom he clashed. He moved to Vienna with his family in 1888, and studied at the Vienna Conservatory (composition with Robert Fuchs, cello with Ferdinand Hellmesberger and theory (the counterpoint class) with Anton Bruckner), graduating "with excellence" in 1896.
He beat 13 other applicants and obtained a post as cellist with the Vienna Court Opera Orchestra, where he played until 1914, often under Gustav Mahler. Mahler habitually had Schmidt play all the cello solos, even though Friedrich Buxbaum was the principal cellist. Schmidt was also in demand as a chamber musician. Schmidt and Arnold Schoenberg maintained cordial relations despite their vast differences in style. Also a brilliant pianist, in 1914 Schmidt took up a professorship in piano at the Vienna Conservatory, which had been recently renamed Imperial Academy of Music and the Performing Arts. (Apparently, when asked who the greatest living pianist was, Leopold Godowsky replied, "The other one is Franz Schmidt.") In 1925 he became Director of the Academy, and from 1927 to 1931 its Rector.
As teacher of piano, cello and counterpoint and composition at the Academy, Schmidt trained numerous musicians, conductors and composers who later achieved fame. Among his best-known students were the pianist Friedrich Wührer and Alfred Rosé (son of Arnold Rosé, the legendary founder of the Rosé Quartet, Konzertmeister of the Vienna Philharmonic and brother-in-law of Gustav Mahler). Among the composers were Theodor Berger, Marcel Rubin and Alfred Uhl. He received many tokens of the high esteem in which he was held, notably the Franz-Josef Order, and an Honorary Doctorate from the University of Vienna.
Schmidt's private life was in stark contrast to the success of his distinguished professional career, and was overshadowed by tragedy. His first wife Karoline Perssin (c1880-1943) was confined in the Vienna mental hospital Am Steinhof in 1919, and three years after his death was murdered under the Nazi euthanasia program. Their daughter Emma Schmidt Holzschuh (1902-1932, married 1929) died unexpectedly after the birth of her first child. Schmidt experienced a spiritual and physical breakdown after this, achieved an artistic revival and resolution in his Fourth Symphony of 1933 (which he inscribed as "Requiem for my Daughter") and, especially, in his oratorio The Book With Seven Seals. His second marriage in 1923, to a successful young piano student Margarethe Jirasek (1891-1964), for the first time brought some desperately needed stability into the private life of the artist, who was plagued by many serious health problems.
Schmidt's worsening health forced his retirement from the Academy in early 1937. In the last year of his life Austria was brought into the German Reich by the Anschluss, and Schmidt was fêted by the Nazi authorities as the greatest living composer of the so-called Ostmark. He was given a commission to write a cantata entitled "The German Resurrection", which, after 1945, was taken by many as a reason to brand him as having been tainted by Nazi sympathy. However, Schmidt left this composition unfinished, and in the summer and autumn of 1938, a few months before his death, set it aside to devote himself to two other commissioned works for the one-armed pianist Paul Wittgenstein, for whom he had often composed: the Quintet in A major for Piano left-hand, Clarinet, and String Trio; and the solo Toccata in D minor. Schmidt died on 11 February 1939.
As a composer, Schmidt was slow to develop, but his reputation, at least in Austria, saw a steady growth from the late 1890s until his death in 1939. In his music, Schmidt continued to develop the Viennese classic-romantic traditions he inherited from Schubert, Brahms and his own master, Bruckner. He also takes forward the exotic "gypsy" style of Liszt and Brahms. His works are monumental in form and firmly tonal in language, though quite often innovative in their designs and clearly open to some of the new developments in musical syntax initiated by Mahler and Schoenberg. Although Schmidt did not write a lot of chamber music, what he did write, in the opinion of such critics as Wilhelm Altmann, was important and of high quality. Although Schmidt's organ works may resemble others of the era in terms of length, complexity, and difficulty, they are forward-looking in being conceived for the smaller, clearer, classical-style instruments of the Orgelbewegung, which he advocated. Schmidt worked mainly in large forms, including four symphonies (1899, 1913, 1928 and 1933) and two operas: Notre Dame (1904-6) and Fredigundis (1916–21). A CD recording of Notre Dame has been available for many years, starring Dame Gwyneth Jones and James King.