Skin color is perceived through the lens of a mental algorithm that initially arose for gender recognition. People can tell whether a face is male or female even if the image is blurred and provides no other useful information than its hue and luminosity.
Today, skin color has a strong ethnic meaning. It differs mostly between people of different geographic origins, at least in modern Western societies. But that has not always been the case. In earlier societies, where people were overwhelmingly of local origin, skin color had a gendered meaning: it differed primarily between men and women.
Women are universally the fair sex (Frost and van den Berghe 1986). They are paler than men, who conversely are ruddier and browner. That sex difference was noticed in earlier times. Wherever the visual arts developed—ancient Egypt, the Greco-Roman world, early South and East Asia, Mesoamerica—female figures were depicted with a lighter hue and male figures with a darker one.
link
The original meaning of skin color — Peter Frost
American Buddhist Net 