NeoVim : Can we "insert literally" every ASCII symbol with `<c-v>` ? (including delete `^?` )

Question

In insert or command mode, the printable ASCII chars, like a can be insert with just a (we don't need <c-v> for those ones, but most of them will still work with <c-v>letter ).

If I type, let's say, <c-v><c-a>, in insert mode, it will put a ^A with the SpecialKey color, meaning it's actually the char with hex value 01 (we can check this with ga), and it's the case for most <c-v><c-letter> shortcuts : It usually puts the ASCII char with value letter modulo 32, so, it's supposed to let me write almost every ASCII chars.

I still see 4 main exceptions to this rule :

1. <c-v><c-@> (or <c-v><c-space>) will put the chars <c-space> (several letters), and not ^@ as I expected.
   There is actually a simple way to write ^@ : <c-v><c-j>.
   
2. So, I have no way to write ^J, but I guess ^J is used to the "visual" newlines in NeoVim :
   I checked with xxd on an empty file, doing : i00000000: 0a0a<Esc>:.!xxd -r<c-m> gives me 2 lines, meaning the 0ath ASCII char is used for that. So, I think there is no way to see a ^J (or only instead of newline with some option), am I right ? In a certain sense, we can say that <c-m> or <Enter> alone is a way to insert the char with hex value 0a.
3. Either <c-i> (tabulation) or <c-v><c-i> display a tabulation char, but the tabulation isn't displayed as <c-i>. It's only displayed as some chars with variable length, depending on options like listchars or tabstop. I don't know if there is a way to display it as ^I instead of "a large space", (but this is not my main question).
4. For delete : <c-v><del> will print <del> and not ^? as I expected. This one is the more problematic, because I haven't found something simple to insert the char with hex value 7f (ie the last ASCII char).

I tried the same with vim, and the behavior differ for 1. and 4. :
with vim, I can type every ASCII char easily, while I can't with neovim.

I know the ASCII delete char can be typed in a harder way, with <c-v>x7f. It's a bit long to do.

Is there a reason why exactly one ASCII char (the delete) can't be inserted in text in a simple way ?

main question: Is there somewhere an option I can configure to tell neovim to get the literal ^? instead of its name <del> ?
(I mean something other than explicitly remapping the shortcuts sequence and explicitly describing the hex value I want, else, the command inoremap <c-v><del> <c-v>x7f can in some sense solve my problem).

I saw some people has almost what I search, but didn't wanted it (for this case, it was without pressing <c-v>). I am pretty sure they didn't explicitly remapped something to <c-v>x7f, or at least, this kind of post let me think there could be an option.
In other words, can we chose somewhere how key names (like <F1>, or <c-left> or anything) are displayed after <c-v> ?

---

Note : I tested in several contexts (both vim and neovim, and each of them with 4 terminals : st, gnome-terminal, konsole and guake, so, at least it's not an "issue" related to one specific terminal) and got exactly the same result every time I do this with neovim. Versions are :

• neovim : 0.4.3
• vim : 8.1.2269

Note : I also tried with zsh shell alone, because there is a similar feature : <c-v><BS> writes ^? in zsh (and it also writes ^? in vim if I edit the shell command with vim, using <c-x><c-e> shortcut from zsh). Also, <c-v><c-@> writes ^@ in zsh, so, if there is a way to tell vim to behave like zsh for this part, it will be a little better for me.

Note : To get ^? with zsh, I needed to use the Backspace key, not the Delete key, while ^? is indeed called DEL when I search it in man 7 ascii. <c-v><del> on zsh writes ^[[3~ (an Escape or ^[, followed by 3 printable chars). Same for other shells (at least bash). Same for vim. So, I also tried <c-v><BS> in neovim, but it only writes <BS> as text, neither ^h nor ^?.

Edit : I discovered that my vim command is /usr/bin/vim, but it's actually a symlink to a symlink to /usr/bin/nvim, so, it ran neovim instead. I need to explicitly type /usr/bin/vim.basic to launch the true vim. In vim only, the <c-v><BS> writes ^? as I wanted. So, I guess my problem is related neovim only. I edited this in my text. Now, when I write vim, I'm actually referring to the command /usr/bin/vim.basic that launches it (and not the vim command that launched neovim). I also added the neovim tag.