Try unsetting 't_ks' and 't_ke' in your vimrc:
" Prevent Vim from setting application keypad mode.
set t_ks= t_ke=
If you'd like, you can do that conditionally, based on the TERM=vt520 setting:
" Prevent Vim from setting application keypad mode on a VT520.
if &term ==# 'vt520'
set t_ks= t_ke=
endif
These sequences are used by Vim to start and end "keypad transmit" mode.
Vim documentation also mentions those in this help section and, in particular, recommend disabling them (setting them to an empty string) in cases where they cause trouble:
Some termcap entries are wrong in the sense that after sending 't_ks' the
cursor keys send codes different from the codes defined in the termcap. To
avoid this you can set 't_ks' (and 't_ke') to empty strings. This must be done during initialization, otherwise it's too late.
$TERMset tovt520? Or to something else likextermorscreenorlinuxorvt100? Please edit the question and include the output ofecho $TERM(in your shell) or:set term?(in Vim). Thanks!