There are clothes that flatter, accentuate and add to a persons appearance. Also there are articles of clothing that are unflattering, distracting and unattractive for certain individuals and figures, both male and female. As in does this blouse make me look fat? I can't wear "polo shirts". Maybe mostly because I feel they make me look short and dumpy. If I feel I do not look like me or do not look my best, then I do not. This is typically much more important to women.
There are women , and I suppose men also that do the most with what they have. There was a woman at work ,"Nina". At first look she was a complete "babe", great figure, shapely legs, high firm breasts, a real attractive woman. When I, the male, got over my first appraisal of her feminine charms I came to a different conclusion. Nina knew how to work with what she had. Her make up, hair, the clothes she wore were all carefully picked to accentuate her best attributes. She was an exciting attractive woman but she worked at it.
I recently read an article on a leading university study that concluded that women felt their most attractive when dressed up made up and ready to go out. Men on the other hand found women to be their most attractive when dressed down. Men preferred tee shirt, jeans, and pony tail. Men found women more attractive when they were dressed for every day, they looked more accessible.

Clothes? Why wear any!

OK - OK....just playing. For me it is a combination of the person making the clothes work and the clothes making the person. The person's taste, their mood, their attitude is what makes it all come together.