Sorry to say but you've got it backwards. Colors correspond to certain wavelengths of light from the visible spectrum. Black is the absence of light and therefore absent of color. And white (light) contains all of the colors in the spectrum, however it is not a color itself. But of course this is all just technical and most people refer to black and white as "colors" casually.
As for the question of the topic, I like both black and white clothing but I think I would pick black over white more often because white is easy to stain, more prominently shows blemish (like sweatiness), or it is more often see-through.