The default encoding seems to be
When entering characters, Vim tries to convert them from the incoming character set (probably unicode) to latin-1. This fails since latin-1 doesn't have these characters. Hence the question marks.
Opening an existing file (Saved with notepad or some other program) should result in garbled text as now Vim is just trying to read bytes and interpret them according to the latin-1 character set, and not convert them to the latin-1 set.
You will want to use
utf-8 or some such:
After which Malayalam script seems to work.
Note that this will not make pre-existing question marks work. These really are converted to question marks (character
0x3f) on input. There is no way to get back what was entered. I suspect that this is the source of confusion of this question.
As a sidenote, unsupported glyphs in the font are usually rendered with a distinct glyph; this varies per font, but is typically either a square block or a different styled question mark. you can see the difference between a "real" question mark and an unsupported glyph with
g8, which will show the character code.