He was the eldest son of Lieutenant-General Anne de Tubières, comte de Caylus. His mother, Marthe-Marguerite de Villette de Mursay, comtesse de Caylus (1673–1729), was the daughter of vice-admiral Philippe, Marquis de Villette-Mursay.
He was a cousin of Mme de Maintenon, who brought Marthe-Marguerite up like her own daughter. Marthe-Marguerite wrote valuable Souvenirs of the court of Louis XIV; these were edited by Voltaire (1770), and by many later editors.
While a young man, Caylus distinguished himself in the campaigns of the French army, from 1709 to 1714. After the peace of Rastatt (1714) he spent some time in travelling in Italy, Greece, the Levant, England and Germany, and devoted much attention to the study and collection of antiquities. He became an active member of the Académie royale de peinture et de sculpture and of the Académie des Inscriptions. Chief among his antiquarian works must be the profusely illustrated Recueil d'antiquités égyptiennes, étrusques, grecques, romaines et gauloises (6 vols., Paris, 1752-1755), which was mined by the designers of Neoclassical arts for the rest of the century. His Numismata Aurea Imperatorum Romanorum, treats only the gold coinage of the Roman emperors, those worthy of collection by a grand seigneur. His concentration on the object itself marked a step towards modern connoisseuship, and in his Mémoire (1755) on the method of encaustic painting, the ancient technique of painting with wax as a medium mentioned by Pliny the Elder, he claimed to have rediscovered the method. Diderot, who was no friend to Caylus, maintained that the proper method had been found by J.-B. Bachelier.
Caylus was an admirable etcher, and copied many paintings of the great masters. He caused engravings to be made, at his own expense, of Bartoli's copies from ancient pictures and published Nouveaux sujets de peinture et de sculpture (1755) and Tableaux tirés de l'Iliade, de l'Odyssée, et de l'Enéide (1757).
He encouraged artists whose reputations were still in the making, and befriended the connoisseur and collector of prints and drawings Pierre-Jean Mariette when Mariette was only twenty-two, but his patronage was somewhat capricious. Diderot expressed this fact in an epigram in his Salon of 1765: "La mort nous a délivré du plus cruel des amateurs." Caylus had quite another side to his character. He had a thorough acquaintance with the gayest and most disreputable sides of Parisian life, and left a number of more or less witty stories dealing with it. These were collected (Amsterdam, 1787) as his Œuvres badines complètes. The best of them is the Histoire de M. Guillaume, cocher (c. 1730). His Contes, hovering between French fairy tales and oriental fantasies, between conventional charm and moral satire, have been collected and were published in 2005.
The Souvenirs du comte de Caylus, published in 1805, is of very doubtful authenticity. See also E. and J. de Goncourt, Portraits intimes du XVIIIième siècle; Charles Nisard's edition of the Correspondance du comte de Caylus avec le père Paciaudi (1877); and a notice by O. Uzanne prefixed to a volume of his Facties (1879).