Because you're telling vim to perform some commands from a file you may not have control over or someone else could have written. Say you grab something from github, and deep in the tree there's a .vimrc. If you vim something in that directory and have
set exrc, you're telling vim to run the commands in the .vimrc you got from github. Do you completely trust whoever created that file? Because you're letting vim run whatever commands they put in that file. Or say you're on a machine regularly used by multiple people (they still exist :-) ), and someone asks you to look at something in their home directory (or /tmp, or any place they can write to). You cd there, it's a text file, so you do a
vim file. With
set exrc, you're also telling vim to run the commands in .vimrc in that directory. Do you completely trust that user, or in a world writable directory like /tmp, every user who can log in to that machine?
set secure mitigates the problem somewhat by limiting some things that can be done in a .vimrc, but if the per-directory .vimrc can made to be owned by you (such as in the github example, or on a multi-user machine where some users can chown files to you),
set secure is ignored, so you still have the same problems.