Question
In MacVim I type :tabe jäderberg.txt
to open a buffer for that file. The file does not yet exist. My status line displays the file name jäderberg.txt
. I do i<C-r>%
to put the file name into the buffer and it puts jäderberg.txt
. I write and quit the buffer.
Now I run MacVim from Terminal by typing mvim j<Tab><CR>
. The file opens, but the statusline displays ja¨derberg.txt
. I type i<C-r>%
and i get ja¨derberg.txt
. If in Terminal I type out the whole file name, mvim jäderberg.txt
, everything is fine. If I type mvim jä<Tab>
Terminal refuses to complete the file name.
Why is my status line not displaying the file name correctly when I open MacVim from Terminal and tab-complete the filename?
As far as I've gotten
"Accented characters" can be stored as a single code point or as a series of code points. I don't know much about this, but I can tell there is a difference between how Terminal treats the character ä
that I type and the character ä
that is stored in a file name and completed with <Tab>
.
Typing jä
:
$ printf jä | xxd
0000000: 6ac3 a4 j..
Completing jä
:
$ touch jä
$ printf jä | xxd
0000000: 6a61 cc88 ja..
This seems to correspond to a difference between composed and decomposed characters. I don't understand why Vim should fail to correctly display the decomposed ä
that is actually stored in the file name, and only when I give the file name with a decomposed ä
.
Observations
Here are some further observations. I'm not sure what other info is relevant, so ask and I'll add.
If I move the cursor over the characters
ja¨derberg.txt
in the buffer, they change. With the cursor onj
and pressingl
until end of line, here is how the line changes.ja¨derberg.txt ja¨derberg.txt jadderberg.txt jadeerberg.txt jaderrberg.txt ... jaderberg.txtt
I wrote a function to loop through all characters on the line and convert each to a number and back to a char, then write the results to the buffer.
function! JJDebugEncoding(lineno) let line = getline(a:lineno) let newline = "" for char in split(line, '\zs') let nr = char2nr(char) let newline = newline . nr2char(nr) endfor call append(a:lineno, newline) endfunc
When I run it for a line with
ja¨derberg.txt
I getjaderberg.txt
.There is no byte value
228 (ä)
or168 (¨)
anywhere in the buffer.Terminal Vim does not seem to have this problem.
encoding
is set toutf-8
. I'm not sure where this option is set sincelatin1
is the default, I'm not setting it in my vimrc and:verbose set encoding
doesn't tell me anything. But I have tested that it is already set by the time my vimrc is sourced (that's why I don't set it myself) so I figure that should be ok.fileencoding
defaults toencoding
if it is not set, but I have also tried setting it explicitly before writing the buffer to file with no difference.fileencodings
is set toucs-bom,utf-8,default,latin1
andfileformats
tounix,dos,mac
.
Edit:
a.) NFS+
requires file names to only contain characters in Unicode Normal Form D, that is, accented accented characters are decomposed. Other filesystems seem to favor but not enforce UNF-C, composed. Therefore this is likely a MacVim-specific problem.
b.) I was wrong to say this does not affect terminal Vim. It does, and ga
with cursor on ä
gives
" typed
<a> 97, Hex 61, Octal 141 < ̈> 776, Hex 0308, Octal 1410
" from % register or tab-completed
<ä> 228, Hex 00e4, Octal 344
The reason I didn't notice this difference is that terminal Vim renders the characters the same way.
c.) Some relevant and helpful reading
convmv --nfc -t utf-8 -f utf-8 j*derberg.txt
does it work?