Vim hex editor with xxd not working as intended

Question

I'm trying to use Vim as a hex editor.

I am using the %!xxd / %!xxd -r method, and it worked fine until I tried a some bytes that didn't work as expected :

00000000: 1337 6161 6161 6161 6161 6161 6161 6161  .7aaaaaaaaaaaaaa
00000010: 6161 6161 6161 6161 6161 6161 6161 62ff  aaaaaaaaaaaaaabR
00000020: 3713 0000 0a                             7....

With this in the Vim buffer, running %!xxd -r and then %!xxd will produce :

00000000: 1337 6161 6161 6161 6161 6161 6161 6161  .7aaaaaaaaaaaaaa
00000010: 6161 6161 6161 6161 6161 6161 6161 62c3  aaaaaaaaaaaaaab.
00000020: bf37 1300 000a                           .7....

Notice the difference at the end of the second line.

Now you might think that xxd is not used correctly, however running this : cat bytes | xxd -r | xxd will yield :

00000000: 1337 6161 6161 6161 6161 6161 6161 6161  .7aaaaaaaaaaaaaa
00000010: 6161 6161 6161 6161 6161 6161 6161 62ff  aaaaaaaaaaaaaab.
00000020: 3713 0000 0a                             7....

which is what I was expecting to see in Vim.

Do you have any idea of what I am doing wrong?

Answer 1

This will happen if you start editing the xxd-dumped version of the binary file, in which case Vim will assume a UTF-8 encoding (which you can confirm with :set fenc? returning fileencoding=utf-8).

When you turn the file back into binary form, the character still looks correct (using ga on top of it shows <ÿ> 255, Hex 00ff, Oct 377, Digr y:), but that's actuall not considering the encoding, so if you were to save this file now, the UTF-8 encoding will kick in and will turn \u00ff into the bytes \xc3\xbf, which are the UTF-8 encoded version of this Unicode character.

One possible workaround is to use a different file encoding such as latin1, which is a 8-bit encoding and shouldn't expand characters above 127.

A better solution is to use vim -b to enable the 'binary' option. (You can also use :set binary from inside Vim.) Enabling 'binary' will disable a file encoding completely (:set fenc? returns fileencoding=) and should preserve 8-bit characters as they are.

It turns out this works OK with the xxd-dumped output since xxd only uses 7-bit ASCII characters in its output, so using 'binary' doesn't really hurt it too much.

(From a purist's point of view, an ideal solution here would use two separate buffes in Vim, one for the xxd-dumped output and a separate one with 'binary' set for the binary encoded output of xxd. A plug-in facilitating binary editing would probably help manage the separate buffers and set the right options on all of them.)

Answer 2

Here's what I have in my ~/.vimrc to allow quite comfortable and issue-free editing of binary files, which are narrowed down to *.bin and *.hex files in the vimcode below, but that can be adjusted as needed, of course:

" Use xxd(1) to make editing binary files actually usable, by filtering
" the entire buffer through xxd(1) as an external command, and save/restore
" the view position while saving the buffer, to stay at the same position
"
" NOTE: Thanks to the file's dual nature, restoring the last saved position
"       of the cursor upon opening a file this way is effectively broken
"
augroup vimrc-xxd-binary-edit
  autocmd!
  autocmd BufReadPre  *.bin,*.hex set binary
  autocmd BufReadPost *.bin,*.hex
    \ if &binary
    \ |   execute "silent %!xxd -autoskip -cols 32"
    \ |   set filetype=xxd
    \ |   redraw
    \ | endif
  autocmd BufWritePre *.bin,*.hex
    \ if &binary
    \ |   let s:winview = winsaveview()
    \ |   execute "silent %!xxd -revert -autoskip -cols 32"
    \ | endif
  autocmd BufWritePost *.bin,*.hex
    \ if &binary
    \ |   execute "silent %!xxd -autoskip -cols 32"
    \ |   set nomodified
    \ |   call winrestview(s:winview)
    \ |   redraw
    \ | endif
augroup END

As noted in the code comments above, you may also want to disable restoring of the cursor position upon opening a file when &filetype ==# "xxd". See also :h restore-cursor for the usual way the cursor position is restored.

See also my patches that contain both the vimcode visible above and more related vimcode, which have been accepted upstream by the vim project in commits 6a500661a9 and 81b8bf5b4a.