Literally translated as, mixing dishtowels with napkins, the expression means mixing things of different kinds, with the implication that some of those things are superior to the others. It is often used in the negative form, as in il ne faut pas mélanger les torchons et les serviettes, to express one should treat things or people differently according to their perceived value or class. A roughly equivalent English expression could be you should separate the sheep from the goats, with the added notion of one being better than the other one.
It is a colloquial expression that can be delivered either earnestly or ironically, to deride a person’s or an institution’s narrowmindedness.
This expression first appeared in the 16th century, at a time when only "bourgeois" would ever use napkins. It relies on the symbolic opposition between the dishtowel, seen here as a lowly rag used for domestic chores, and the napkin, a much more distinguished piece of cloth that is an integral part of an elegant table setting. The classist — though now outdated — implication was that the former was in the realm of servants, while the latter belonged to the world of their employers and their social life. It would then have been improper to wash or put away the two together.
Ranger un polar avec la littérature classique? C'est mélanger les torchons avec les serviettes!
To put away a mystery book with classic literature? You've got to know what's what!
Les meubles anciens ne vont pas avec les meubles modernes; il ne faut pas mélanger les torchons et les serviettes.
Antiques don't fit in with modern furniture; it's like mixing apples and oranges.