This an odd question, and would be equally odd if asked of any editor IMHO.
No, vim (nor any other vi-variant) has ever hindered any of my work. I've only ever been hindered when not using it.
I'm not sure how vim would hinder my social interaction with other team members. It's not like vim gives you bad breath.
My colleagues and bosses couldn't care less what editor I use. About 70% of them use vim/vi too, a handful use emacs, most of the GUI developers use Eclipse, and one or two might use something else like jEdit or Notepad++. They don't care what brand of pen I take notes with either.
Also, I don't see how vim forces you to configure it. Out of the box, it works pretty much like any version of vi ever has. Yeah, you end up tweaking it to your liking over time, but probably less than any IDE.
I haven't done any official pair programming, although two of useus sometimes do work together to solve a particularly thorny problem. But then again, I don't see that the choice of editor matters much (everyone here except perhaps some of the GUI developers at least know how to use vim/vi, even if it's not their editor of choice).