Why do we consider vanilla as neutral?

When talking about sweets like ice cream or unmodded computer games like Minecraft, "vanilla" is often used as a synonym for "neutral" or "usual".
Why is that so? Why isn't chocolate "neutral"? Is it somehow related to the color white?

According to the Oxford English Dictionary, this sense of "vanilla" goes back only to 1972, which surprised me; I'd have thought it was older. And the earliest uses of it referred to straight and non-kinky sexual preferences. The Hacker's Dictionary documented it in its computer-related sense in 1983.

The OED says it comes from "the popular perception" of vanilla as the "ordinary, bland flavour" of ice cream.

Vanilla is a flavoring that is widely used in anything sweet. How often do you come across a dessert recipe that doesn't include vanilla? And look at the ingredients list of any packaged sweet treat; you will almost always see vanillin (the chemical that gives vanilla most of its flavor) listed. Or vanilla itself, but usually they're cheap and use vanillin.

Ever eaten a dessert that had too much vanilla? No? I haven't, either. I have, however, had desserts that had too much chocolate, too much lemon, too much cinnamon, etc and the flavor was overpowering.

Vanilla is a neutral flavor that goes well with everything and doesn't overpower.

(As a side point, I think people really ought to be more careful about synonymizing "neutral"/"plain" with "vanilla." For example, you don't want to go buy a plain yogurt if what you're really looking for is vanilla!)