120

Is there a way to edit binary files in some kind of hexadecimal mode?

For example if I have some binary data shown by xxd or hexdump -C like this:

$ hexdump -C a.bin | head -n 5
00000000  cf fa ed fe 07 00 00 01  03 00 00 80 02 00 00 00  |................|
00000010  12 00 00 00 40 05 00 00  85 00 20 00 00 00 00 00  |....@..... .....|
00000020  19 00 00 00 48 00 00 00  5f 5f 50 41 47 45 5a 45  |....H...__PAGEZE|
00000030  52 4f 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |RO..............|
00000040  00 00 00 00 01 00 00 00  00 00 00 00 00 00 00 00  |................|

$ xxd a.bin | head -n 5
0000000: cffa edfe 0700 0001 0300 0080 0200 0000  ................
0000010: 1200 0000 4005 0000 8500 2000 0000 0000  ....@..... .....
0000020: 1900 0000 4800 0000 5f5f 5041 4745 5a45  ....H...__PAGEZE
0000030: 524f 0000 0000 0000 0000 0000 0000 0000  RO..............
0000040: 0000 0000 0100 0000 0000 0000 0000 0000  ................

If I want to change the value at a specific position, this kind of view will help finding the right place, for example when the position to change is near some known string.

0

9 Answers 9

127

The simplest way is to use the binary option. From :help binary:

This option should be set before editing a binary file.  You can also
use the -b Vim argument.  When this option is switched on a few
options will be changed (also when it already was on):
        'textwidth'  will be set to 0
        'wrapmargin' will be set to 0
        'modeline'   will be off
        'expandtab'  will be off
Also, 'fileformat' and 'fileformats' options will not be used, the
file is read and written like 'fileformat' was "unix" (a single <NL>
separates lines).
The 'fileencoding' and 'fileencodings' options will not be used, the
file is read without conversion.

[..]

When writing a file the <EOL> for the last line is only written if
there was one in the original file (normally Vim appends an <EOL> to
the last line if there is none; this would make the file longer).  See
the 'endofline' option.

If you don't do this, and your environment is using a multibyte encoding (e.g. UTF-8, as most people use), Vim tries to encode the text as such, usually leading to file corruption.

You can verify this by opening a file, and just using :w. It is now changed.
If you set LANG and LC_ALL to C (ASCII), Vim doesn't convert anything and the files stay the same (it still adds a newline, though) since Vim won't need to do any multibyte encoding.

I personally also prefer to disable set wrap for binary, although others might prefer to enable it. YMMV. Another useful thing to do is :set display=uhex. From :help 'display':

uhex            Show unprintable characters hexadecimal as <xx>
                instead of using ^C and ~C.

And as a last tip, you can show the hex value of the character under the cursor in the ruler with %B (:set rulerformat=0x%B).

More advanced: xxd

You can use the xxd(1) tool to convert a file to more readable format, and (this is the important bit), parse the edited "readable format" and write it back as binary data. xxd is part of vim, so if you have vim installed you should also have xxd.

To use it:

$ xxd /bin/ls | vi -

Or if you've already opened the file, you can use:

:%!xxd

Now make your changes, you need to do that on the left-hand side of the display (the hex numbers), changes to the right-hand side (printable representation) are ignored on write.

To save it, use xxd -r:

:%!xxd -r > new-ls

This will save the file to new-ls.

Or to load the binary in the current buffer:

:%!xxd -r

From xxd(1):

   -r | -revert
          reverse operation: convert (or patch) hexdump into  binary.   If
          not  writing  to stdout, xxd writes into its output file without
          truncating it. Use the combination -r -p to read plain hexadeci‐
          mal dumps without line number information and without a particu‐
          lar column layout. Additional  Whitespace  and  line-breaks  are
          allowed anywhere.

And then just use :w to write it. (beware: you want to set the binary option before you write to the file, for the same reasons outline above).

Complementary keybinds to make this a bit easier:

" Hex read
nmap <Leader>hr :%!xxd<CR> :set filetype=xxd<CR>

" Hex write
nmap <Leader>hw :%!xxd -r<CR> :set binary<CR> :set filetype=<CR>

This is also available from the menu if you're using gVim, under 'Tools ➙ Convert to HEX' and 'Tools ➙ Convert back'.

The vim tips wiki has a page with more information and some helper scripts. Personally, I think you're probably better off using a real hex editor if you're editing binary files that often. Vim can sort of do the job, but it's obviously not designed for it, and if you ever write without :set binary Vim might destroy your binary files!

6
  • 6
    Lovely answer, but should probably start with "Don't try this at home, kids!"
    – msw
    Commented Oct 22, 2016 at 23:24
  • What if I need to remove some bytes? e.g. in the middle of binary.
    – Anton K
    Commented Nov 17, 2017 at 13:59
  • I don't know what Vim is doing, but it's adding 95KB of text to a 200KB binary file despite me having changed nothing. Even with :set binary noeol fenc=utf-8. In fact, it's doing it immediately upon opening the file before it says [noeol] [converted]. Why does vim need to make the buffer 150% bigger? How do I prevent it from corrupting files like that? Commented Apr 1, 2018 at 19:35
  • 1
    The only thing that works is :r !xxd <file> (or $ xxd <file> | vim -) to read, and :w !xxd -r > <file> to write, but this is not ideal. Commented Apr 1, 2018 at 19:46
  • Excellent answer. Note that the url for bless does not work; I found it (I think) on github at github.com/bwrsandman/Bless.
    – sonofagun
    Commented Jan 10, 2019 at 1:25
33

To view the content of a binary file in a hex view, open the file, switch on binary mode, and filter the buffer through the xxd command:

:set binary
:%!xxd

You can make changes in the left area (edit the hex numbers), and when ready, filter through xxd -r, and finally save the file:

:%!xxd -r
:w

If the filtering step after opening and before closing sounds tedious, and you often do this with files with .bin extension, you can add this to your vimrc to make the process automatic:

" for hex editing
augroup Binary
  au!
  au BufReadPre  *.bin let &bin=1
  au BufReadPost *.bin if &bin | %!xxd
  au BufReadPost *.bin set ft=xxd | endif
  au BufWritePre *.bin if &bin | %!xxd -r
  au BufWritePre *.bin endif
  au BufWritePost *.bin if &bin | %!xxd
  au BufWritePost *.bin set nomod | endif
augroup END
3
  • If I follow these instructions (open binary file, :%!xxd, :%!xxd -r, :w, didnt make any changes!) then the binary file written is not the same as the original ... Is this the case for you (I tested with /bin/ls). I need to use :set binary before saving (also see my answer which explains why) ... Maybe it's something in my vimrc? But regardless, I would always use set binary for safety ... Commented Feb 4, 2015 at 21:31
  • 1
    You may instead add the augroup script to ~/.vim/plugin/binary.vim if you don't wish to clutter your .vimrc
    – thom_nic
    Commented Sep 6, 2017 at 16:00
  • If you're on a foreign install, that augroup Binary listing is located at :help hex-editing or :help using-xxd in any Vim since 5.5 (Sep 1999).
    – bb010g
    Commented Aug 15, 2019 at 23:34
13

TL;DR Answer

Open the file with Vim in binary mode:

vim -b <file_to_edit>

In Vim, get into hex editing mode like so:

:%!xxd -p

To save:

:%!xxd -p -r
:w

The first line will convert the buffer back from hex mode and the second line will save the file like normal.

Note the -p option. This avoids all the extra printable and address fluff and just shows you the hex. Just omit -p if you want the extra context.

Be careful opening the file with Vim not in binary mode, as it will append a (usually unintended) LF character to the end of the file when you save it.

3
  • This doesn't really add anything that's not in the other answers.
    – Herb
    Commented Mar 29, 2017 at 4:17
  • 6
    The real TL;DR is in :h using-xxd and has been around since v7.0001 and probably longer. This site would be less active if people searched the docs.
    – Tommy A
    Commented Mar 30, 2017 at 13:15
  • I tried this both ways and you have to do the -b or :set binary when editing the file or it will booger up your file with the LF at the end. I used :%!xxd -g 1 because that is the format I am used to seeing. xxd -r is bright enough to deal with the result with or without the -g 1 Commented Jul 29, 2022 at 20:55
10

Use the "bvi" editor. http://bvi.sourceforge.net/ (It's in every Linux repository.)

$ apt-cache show bvi
[snip]
Description-en: binary file editor
 The bvi is a display-oriented editor for binary files, based on the vi
 text editor. If you are familiar with vi, just start the editor and begin to
 edit! If you never heard about vi, maybe bvi is not the best choice for you.
2
2

This looks like a handy little vim plugin that does the job using a temporary file which it writes back and forth for you automatically.

Some years ago I found a similar plugin which I adapted and improved for my own use. I've included the relevant code for that here, in case anyone wants it. It too is based on the xxd tool. I'm sure the GitHub version I linked above works better, but I haven't actually used it myself, so I figured I'd also post this one which I know for sure works.

The source for this other version was the vim wikia, specifically this page.

Here's the code:

"-------------------------------------------------------------------------------  
" Hexmode  
"-------------------------------------------------------------------------------  
" Creates an automatic hex viewing mode for vim by converting between hex dump  
" and binary formats. Makes editing binary files a breeze.  
"-------------------------------------------------------------------------------  
" Source: vim.wikia.com/wiki/Improved_Hex_editing  
" Author: Fritzophrenic, Tim Baker  
" Version: 7.1  
"-------------------------------------------------------------------------------  
" Configurable Options {{{1  
"-------------------------------------------------------------------------------  

" Automatically recognized extensions  
let s:hexmode_extensions = "*.bin,*.exe,*.hex"  

"-------------------------------------------------------------------------------
" Commands and Mappings {{{1
"-------------------------------------------------------------------------------

" ex command for toggling hex mode - define mapping if desired
command! -bar Hexmode call ToggleHex()
command! -nargs=0 Hexconfig edit $VIM\vimfiles\plugin\hexmode.vim | exe "normal 11G" | exe "normal zo"

nnoremap <C-H> :Hexmode<CR>
inoremap <C-H> <Esc>:Hexmode<CR>
vnoremap <C-H> :<C-U>Hexmode<CR>

"-------------------------------------------------------------------------------    
" Autocommands {{{1  
"-------------------------------------------------------------------------------  

if exists("loaded_hexmode")  
    finish  
endif  
let loaded_hexmode = 1  

" Automatically enter hex mode and handle file writes properly  
if has("autocmd")  
  " vim -b : edit binary using xxd-format  
  augroup Binary  
    au!  

    " set binary option for all binary files before reading them  
    exe "au! BufReadPre " . s:hexmode_extensions . " setlocal binary"

    " if on a fresh read the buffer variable is already set, it's wrong
    au BufReadPost *
          \ if exists('b:editHex') && b:editHex |
          \   let b:editHex = 0 |
          \ endif

    " convert to hex on startup for binary files automatically
    au BufReadPost *
          \ if &binary | Hexmode | endif

    " When the text is freed, the next time the buffer is made active it will
    " re-read the text and thus not match the correct mode, we will need to
    " convert it again if the buffer is again loaded.
    au BufUnload *
          \ if getbufvar(expand("<afile>"), 'editHex') == 1 |
          \   call setbufvar(expand("<afile>"), 'editHex', 0) |
          \ endif

    " before writing a file when editing in hex mode, convert back to non-hex
    au BufWritePre *
          \ if exists("b:editHex") && b:editHex && &binary |
          \  let oldro=&ro | let &ro=0 |
          \  let oldma=&ma | let &ma=1 |
          \  silent exe "%!xxd -r" |
          \  let &ma=oldma | let &ro=oldro |
          \  unlet oldma | unlet oldro |
          \ endif

    " after writing a binary file, if we're in hex mode, restore hex mode
    au BufWritePost *
          \ if exists("b:editHex") && b:editHex && &binary |
          \  let oldro=&ro | let &ro=0 |
          \  let oldma=&ma | let &ma=1 |
          \  silent exe "%!xxd" |
          \  exe "set nomod" |
          \  let &ma=oldma | let &ro=oldro |
          \  unlet oldma | unlet oldro |
          \ endif
  augroup END  
endif  

"-------------------------------------------------------------------------------
" Functions {{{1
"-------------------------------------------------------------------------------

" helper function to toggle hex mode
function! ToggleHex()
  " hex mode should be considered a read-only operation
  " save values for modified and read-only for restoration later,
  " and clear the read-only flag for now
  let l:modified=&mod
  let l:oldreadonly=&readonly
  let &readonly=0
  let l:oldmodifiable=&modifiable
  let &modifiable=1
  if !exists("b:editHex") || !b:editHex
    " save old options
    let b:oldft=&ft
    let b:oldbin=&bin
    " set new options
    setlocal binary " make sure it overrides any textwidth, etc.
    let &ft="xxd"
    " set status
    let b:editHex=1
    " switch to hex editor
    set sh=C:/cygwin/bin/bash
    %!xxd
  else
    " restore old options
    let &ft=b:oldft
    if !b:oldbin
      setlocal nobinary
    endif
    " set status
    let b:editHex=0
    " return to normal editing
    %!xxd -r
  endif
  " restore values for modified and read only state
  let &mod=l:modified
  let &readonly=l:oldreadonly
  let &modifiable=l:oldmodifiable
endfunction

" vim: ft=vim:fdc=2:fdm=marker
1

Shameless plug: my own vim-hex allows using :Hex and :Bless to toggle between a hex editor view and the "raw" binary. Make sure to vim -b or :edit ++binary to get the binary option set right, or the file can be corrupted.

1

I have written a hex editor environment for vim that uses xxd to do the conversion in the background. You can find it at http://knockknock.org.uk/hexx.tar.gz and at https://www.vim.org/scripts//script.php?script_id=6053

I call it an 'environment' because it takes-over the normal vim operation (temporarily - you can hop into and out of it at any time); vim really doesn't like binary (not text) files and the plugin has to go to some considerable lengths to stop vim messing things up in some cases

1
  • 1
    Welcome to Vi and Vim! This answer would be improved by demonstrating how the linked code solves the original problem.
    – D. Ben Knoble
    Commented Nov 6, 2023 at 17:01
1

excute :helpgrep USING XXD, then read. This will give you exactly the method suitable to your Vim version.

5
  • Welcome and thank you for writing an answer. Note that consulting :help should be a prerequisite to asking a question (reality, though...). Pure "RTFM" answers are less helpful than showing how the documented behavior can be applied to solve OP's problem. You can always edit to improve your question.
    – Friedrich
    Commented May 31 at 7:38
  • 1
    @Friedrich RE: "consulting :help," please see meta.stackexchange.com/q/391467/389795; the manual requirement is not universal (and not part of our help center).
    – D. Ben Knoble
    Commented May 31 at 13:55
  • I second Friedrich's point, though: this answer lacks substantial useful detail; please edit to improve.
    – D. Ben Knoble
    Commented May 31 at 13:55
  • @D.BenKnoble If you run the command provided, you do can get substantial useful details, and it fits the Vim version you're using perfectly.
    – Lee Li
    Commented Jun 1 at 14:43
  • You are correct, but that's not the point. See vi.stackexchange.com/help/how-to-answer, meta.stackexchange.com/q/7656/389795. A good answer provides supporting details, etc.
    – D. Ben Knoble
    Commented Jun 1 at 15:56
1

After playing a bit with using xxd as a filter for editing binary files, I ended up having to get rid of the dreaded Press ENTER or type command to continue message, which was displayed after opening larger binary files.

Here's what I ended up with in my ~/.vimrc, which differs from the usual approach described in :h using-xxd and already mentioned in another answer:

" Use xxd(1) to make editing binary files actually usable, by filtering
" the entire buffer through an external command
"
augroup vimrc-xxd-binary-edit
  autocmd!
  autocmd BufReadPre   *.bin,*.hex set binary
  autocmd BufReadPost  *.bin,*.hex if &binary | silent %!xxd -autoskip -cols 32
  autocmd BufReadPost  *.bin,*.hex set filetype=xxd | redraw! | endif
  autocmd BufWritePre  *.bin,*.hex if &binary | silent %!xxd -revert -autoskip -cols 32
  autocmd BufWritePre  *.bin,*.hex endif
  autocmd BufWritePost *.bin,*.hex if &binary | silent %!xxd -autoskip -cols 32
  autocmd BufWritePost *.bin,*.hex set nomodified | redraw! | endif
augroup END

See this question and this article for more details about the above-mentioned message and the need to use silent and redraw!. The use of these two commands is also tailored specifically to avoid screen flickering, e.g. when doing :w, which is caused by the buffer being filtered by the external command.

See also this question for more details about the need to use set binary, which actually helped avoid corruption of certain binary files.


Here's an improved version, as a result of playing with filtering through xxd for a few more hours. This version also keeps the current view unchanged after doing :w, which in the version above caused the view to scroll back to the very beginning of the buffer. I believe it's needles to say how annoying and counterproductive that was.

" Use xxd(1) to make editing binary files actually usable, by filtering
" the entire buffer through xxd(1) as an external command, and save/restore
" the view position while saving the buffer, to stay at the same position
"
" NOTE: Thanks to the file's dual nature, restoring the last saved position
"       of the cursor upon opening a file this way is effectively broken
"
augroup vimrc-xxd-binary-edit
  autocmd!
  autocmd BufReadPre  *.bin,*.hex set binary
  autocmd BufReadPost *.bin,*.hex
    \ if &binary
    \ |   execute "silent %!xxd -autoskip -cols 32"
    \ |   set filetype=xxd
    \ |   redraw
    \ | endif
  autocmd BufWritePre *.bin,*.hex
    \ if &binary
    \ |   let s:winview = winsaveview()
    \ |   execute "silent %!xxd -revert -autoskip -cols 32"
    \ | endif
  autocmd BufWritePost *.bin,*.hex
    \ if &binary
    \ |   execute "silent %!xxd -autoskip -cols 32"
    \ |   set nomodified
    \ |   call winrestview(s:winview)
    \ |   redraw
    \ | endif
augroup END

As noted in the code comments above, you may also want to disable restoring of the cursor position upon opening a file when &filetype ==# "xxd". See also :h restore-cursor for the usual way the cursor position is restored.


I'm happy to report that my patches, containing both the vimscript code visible above and more related vimscript code, have been accepted upstream by the Vim project, in commits 6a500661a9 and 81b8bf5b4a.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.