How can I open very large files with good performance?

Question

I have server log files that are several gigabytes in size (on Ubuntu). When I attempt to open them, the terminal locks up for a minute or so while the file is loaded into Vim. Is there a way to reduce this time, for example by setting Vim to load the file on demand, or by some other method?

Answer 1

First, try loading vim with no plugins or vimrc:

vim -u NONE gargantuan.txt

If this is substantially faster, then you probably have syntax highlighting, folding, plugins, or something else going on that's taking up most of the time. Try turning stuff off in your vimrc (and disabling your plugins) until you find the culprit.

Also make sure to set ft= syn= and syntax off.

If this doesn't help, it's probably the case that you have very long lines that are causing the problem. Try set nowrap to turn line wrapping off.

Answer 2

Vim is the wrong tool for the job: you should use a pager like more or less.

If you insist on using an editor, try this example adapted from the Vim wiki:

augroup LargeFile
        let g:large_file = 10485760 " 10MB

        " Set options:
        "   eventignore+=FileType (no syntax highlighting etc
        "   assumes FileType always on)
        "   noswapfile (save copy of file)
        "   bufhidden=unload (save memory when other file is viewed)
        "   buftype=nowritefile (is read-only)
        "   undolevels=-1 (no undo possible)
        au BufReadPre *
                \ let f=expand("<afile>") |
                \ if getfsize(f) > g:large_file |
                        \ set eventignore+=FileType |
                        \ setlocal noswapfile bufhidden=unload buftype=nowrite undolevels=-1 |
                \ else |
                        \ set eventignore-=FileType |
                \ endif
augroup END

Answer 3

The LargeFile.vim plugin is designed to make editing large files faster. See http://www.drchip.org/astronaut/vim/index.html#LARGEFILE .

From the page:

Allows much quicker editing of large files (default: 100MB+ are "large"), at the price of turning off events, undo, syntax highlighting, etc. Also available at vim.sf.net where you may rank it.

According to the manual, the plugin works just by having it installed. You can set the cutoff by changing g:LargeFile, to an integer number of MB, which it says defaults to 20MB (in contrast to the project description which says 100)

The plugin also provides commands :UnLarge, :Large and :Large! to disable, reenable, or force enable (for small files) respectively on the currently loaded file.

Answer 4

I have heard that turning off syntax highlighting can help:

:syntax off

Answer 5

Another pager you may like to try is most: http://www.jedsoft.org/most/

Also, rotate your logs and as this may contribute to less larger logs.