I think it's totally understandable. We establish a pecking order in all other aspects of our lives... subordinate ourselves to some, dominate others... so why not within the bdsm society.

In light of that, perhaps the word itself, switch, is pejoritive... implying that you can turn your two sides on and off at will.

I'm not a switch (so I'm only presuming...) but I imagine it has more to do with how you interact upon meeting a specific person. If they tend to be dominant, do you slide into the submissive role? and if they automatically show their submissiveness to you, do you naturally slide into a dominant role.

Seems perfectly logical and human to me.