Mauve dye was invented in 1856, and became very popular in the 1890s, a decade which was therefore sometimes referred to as 'the mauve decade'.
https://en.wikipedia.org/wiki/Mauve
"Mauve became associated with daring and flaunting of social norms, specifically with decadent art and homosexuality. (By the 1950s, lavender came to symbolize homosexuality, then pink, beginning in the 1970s.)
Well-known figures in the art world during The Mauve Decade were gay, such as author Oscar Wilde and artist Aubrey Beardsley. The latter's sexually explicit prints and engraving shocked the artistic establishment."
http://www.blurtit.com/q394260.html
Whistler's comment was quite possibly a part of his ongoing verbal feud with Oscar Wilde. "He was well-known for his biting wit, especially in exchanges with his friend and rival Oscar Wilde. Both were figures in the Café society of Paris, and they were often the “talk of the town”. They frequently appeared as caricatures in Punch, to their mutual amusement. On one occasion, young Oscar Wilde attended one of Whistler's dinners, and hearing his host make some brilliant remark, apparently said, "I wish I'd said that", to which Whistler riposted, "You will, Oscar, you will!" In fact, Wilde did repeat in public many witticisms created by Whistler.[81] Their relationship soured by the mid-1880s, as Whistler turned against Wilde and the Aesthetic Movement. When Wilde was publicly acknowledged to be a homosexual in 1895, Whistler openly mocked him."
https://en.wikipedia.org/wiki/James_McNeill_Whistler