The only son of a village schoolmaster, Leon Lhermitte's precocious drawing skill won him an annual grant from the state. In 1863, he went to Paris and became a student at the Petite Ecole. Jean-Charles Cazin, a fellow pupil, became a lifelong friend, and Lhermitte later got to know Alphonse Legros, Henri Fantin-Latour, Jules Dalou and Auguste Rodin, who had all studied at the school. In 1864 his charcoal drawing the Banks of the Marne near Alfort was exhibited at the Salon; he continued to exhibit his drawings at the Salon until 1889.
In 1871 Legros introduced him to the dealer Durand-Ruel, who agreed to sell several of his drawings. In 1873, Durand-Ruel sent some of Lhermitte's works to the Dudley Gallery for the first of the annual Black and White exhibitions and Lhermitte subsequently became a regular participant. Lhermitte won a third-class medal in the Salon of 1874 for his painting The Harvest, which was bought by the state. The Tavern, exhibited in the Salon of 1881, initiated the monumental series of paintings on the life of the agricultural worker that came closest to justifying van Gogh's admiring appellation "Millet the Second". The next in the series, Harvesters' Payday became the artist's best-known work.
Lhermitte was a founding member of the Societe Nationale des Beaux-Arts in 1890. In 1894 he was made an officer of the Legion d'honneur. Later, Lhermitte was elected to fill Jacques Henner's chair in painting at the Institut. He continued to exhibit in the first decades of the 20th century, when he was generally seen as a relic of a bygone era, although his style later had an influence on Socialist Realism.
Increasingly he worked in pastel, his draughtsman's skill ever in evidence, producing some sensitive portraits and peasant scenes reminiscent of the earlier and more powerful depictions that van Gogh had cited as "an ideal". His pastels are part of the growing use of pastels in France in the latter half of the nineteenth century, used especially well by the Impressionist group.
Lhermitte was said to have begun experimenting in pastels in 1885, just one year before he exhibited for the first time at the Société des Pastellistes Français held at the Georges Petit gallery in Paris. Here he submitted a dozen pastels which depicted daily life in the areas of Mont-Saint-Père and also his travel to Vittel, Berneval, Laren, and Wissant. Lhermitte was one of the most important proponents for acceptance of the pastel. He continually exhibited his work at these exhibitions and helped forge a new group, that of the pastellistes, where he became a mentor for younger artists.