Yakovlev joined the Vakhtangov Theatre in 1952 but his first flirtation with fame came in 1958, when he played Prince Myshkin in Ivan Pyryev's adaptation of The Idiot. Yakovlev followed his first success with regular appearances in Eldar Ryazanov's comedies, most notably Hussar Ballad (1962), in which he played Poruchik Rzhevsky. The feature was such a resounding success that Rzhevsky's character gave rise to innumerable Russian jokes.
In the 1960s and 1970s Yakovlev's career was varied and interesting, his roles ranging from Stiva Oblonsky in the classic Soviet adaptation of Anna Karenina (1968) to the paranoically jealous Ippolit in another of Ryazanov's comedies, The Irony of Fate (1975). His participation in a series of films about World War II won him the USSR State Prize for 1979.
Yakovlev enjoyed perhaps his greatest popular acclaim in Leonid Gaidai's film version of Mikhail Bulgakov's egregiously funny Ivan Vasilievich Changes His Occupation (also known as Ivan Vasilievich: Back to the Future) (1973). His film career effectively came to a halt after Georgi Daneliya's sci-fi extravaganza Kin-dza-dza!, in which he appeared alongside Yevgeny Leonov.
He performed on the stage of the Vakhtangov Theatre. The actor has also played over seventy roles onstage, including mysterious Casanova (Three Ages of Casanova), brilliant court diplomat Duke Bolingbroke (Glass of Water), and tragically genius Prokofiev (Lessons of Master).