If you and co-worker(s) want to continuously work on the same systems, I'd recommend to use different user accounts, so that each has a different home directory. Unix was designed that way from the start.
If this is just for a short session, and your colleague would be confused by your customizations and remappings, you can quickly start up a vanilla Vim instance with
$ vim -u NONE
This isn't completely identical to a clean Vim installation, because default plugins (like netrw and matchparen) won't be loaded, neither. For that, the following command (best put to an alias) can be used:
$ vim --cmd 'set rtp=$VIM/vimfiles,$VIMRUNTIME,$VIM/vimfiles/after | if exists("+packpath") | let &packpath = &rtp | endif' -N -u NORC -c 'set rtp& | if exists("+packpath") | set packpath& | endif'
If this sharing happens a lot (because you're avid pair programmers), it might be worthwhile to separate your configuration into "standard" plugins (shared) and your personal remappings and tweaks. If you diligently separate these in your
~/.vimrc and surround the latter with
if ! exists('g:sharedconfig'), you can launch a "cleaner" variant via
$ vim --cmd 'let g:sharedconfig = 1'
Going even further, your whole team could put their configurations / home directories on a network share, so that these are accessible from anywhere. This would also help with quickly setting up new systems with your own configuration (which I think is a different question, and has already been covered in depth elsewhere).