Doing it automatically would require you to add something like the following to your vimrc file (source):

au FileType haskell,vhdl,ada let b:comment_leader = '-- '
au FileType vim let b:comment_leader = '" '
au FileType c,cpp,java let b:comment_leader = '// '
au FileType sh,make let b:comment_leader = '# '
au FileType tex let b:comment_leader = '% '
noremap <silent> ,c :<C-B>sil <C-E>s/^/<C-R>=escape(b:comment_leader,'\/')<CR>/<CR>:noh<CR>
noremap <silent> ,u :<C-B>sil <C-E>s/^\V<C-R>=escape(b:comment_leader,'\/')<CR>//e<CR>:noh<CR>

Using ,c to comment a region and ,u to uncomment a region. This manually sets the comment symbols for different languages.

The third option is to use a plugin like tcomment, vim-commentary or comments.vim. I use tcomment myself. Please, read the instructions on usage and installation at their pages, as I beleive that's beyond the topic of the question.

I'd suggest you use a plugin (one of the linked above or another) as that is much easier than maintaining a piece of code in your vimrc file.

Edit: I removed the manual way as the question was changed and also the correct way was answered by 200_success.

Doing it automatically would require you to add something like the following to your vimrc file (source):

au FileType haskell,vhdl,ada let b:comment_leader = '-- '
au FileType vim let b:comment_leader = '" '
au FileType c,cpp,java let b:comment_leader = '// '
au FileType sh,make let b:comment_leader = '# '
au FileType tex let b:comment_leader = '% '
noremap <silent> ,c :<C-B>sil <C-E>s/^/<C-R>=escape(b:comment_leader,'\/')<CR>/<CR>:noh<CR>
noremap <silent> ,u :<C-B>sil <C-E>s/^\V<C-R>=escape(b:comment_leader,'\/')<CR>//e<CR>:noh<CR>

Using ,c to comment a region and ,u to uncomment a region. This manually sets the comment symbols for different languages.

The second option is to use a plugin like tcomment, vim-commentary or comments.vim. I use tcomment myself. Please, read the instructions on usage and installation at their pages, as I beleive that's beyond the topic of the question.

I'd suggest you use a plugin (one of the linked above or another) as that is much easier than maintaining a piece of code in your vimrc file.

Edit: I removed the manual way as the question was changed and also the correct way was answered by 200_success.