27

I have server log files that are several gigabytes in size (on Ubuntu). When I attempt to open them, the terminal locks up for a minute or so while the file is loaded into Vim. Is there a way to reduce this time, for example by setting Vim to load the file on demand, or by some other method?

  • 4
    Why not use less? – Bernhard Feb 3 '15 at 20:33
  • 1
    Do you intend to manipulate the files in any way, or do you just want to view them? – 200_success Feb 3 '15 at 20:33
  • 1
    I would like to edit the files. – peter-b Feb 3 '15 at 20:33
  • related: stackoverflow.com/a/19795855/732016 – wchargin Feb 3 '15 at 20:56
  • Most likely I will get banned from the site forever but I have to say it: when it comes to work with huge files Emacs is the tool for the job. :-) – toro2k Feb 4 '15 at 15:09
25

First, try loading vim with no plugins or vimrc:

vim -u NONE gargantuan.txt

If this is substantially faster, then you probably have syntax highlighting, folding, plugins, or something else going on that's taking up most of the time. Try turning stuff off in your vimrc (and disabling your plugins) until you find the culprit.

Also make sure to set ft= syn= and syntax off.

If this doesn't help, it's probably the case that you have very long lines that are causing the problem. Try set nowrap to turn line wrapping off.

  • 1
    vim -U NONE -N gargantuan.txt will do the same thing while running vim in nocompatible which runs in "vi iMproved" mode instead of trying to be compatible with the old "vi" – ecerulm May 11 '17 at 8:37
15

Vim is the wrong tool for the job: you should use a pager like more or less.

If you insist on using an editor, try this example adapted from the Vim wiki:

augroup LargeFile
        let g:large_file = 10485760 " 10MB

        " Set options:
        "   eventignore+=FileType (no syntax highlighting etc
        "   assumes FileType always on)
        "   noswapfile (save copy of file)
        "   bufhidden=unload (save memory when other file is viewed)
        "   buftype=nowritefile (is read-only)
        "   undolevels=-1 (no undo possible)
        au BufReadPre *
                \ let f=expand("<afile>") |
                \ if getfsize(f) > g:large_file |
                        \ set eventignore+=FileType |
                        \ setlocal noswapfile bufhidden=unload buftype=nowrite undolevels=-1 |
                \ else |
                        \ set eventignore-=FileType |
                \ endif
augroup END
  • Can less or more also make changes to my file? – Martin Tournoij Feb 3 '15 at 21:50
  • 3
    They can't, of course. Logs are not meant to be edited. – romainl Feb 3 '15 at 21:59
  • 5
    sed was invented for this purpose. You figure out which changes you want to make, expressed as substitutions, then run sed to perform them. This also works on streams (i.e. file content that has no end because it is being produced). – reinierpost Jun 1 '15 at 8:30
  • I have aliased less, more and man to vimpager. github.com/rkitover/vimpager – Alex Kroll Jun 6 '15 at 8:16
5

The LargeFile.vim plugin is designed to make editing large files faster. See http://www.drchip.org/astronaut/vim/index.html#LARGEFILE .

From the page:

Allows much quicker editing of large files (default: 100MB+ are "large"), at the price of turning off events, undo, syntax highlighting, etc. Also available at vim.sf.net where you may rank it.

According to the manual, the plugin works just by having it installed. You can set the cutoff by changing g:LargeFile, to an integer number of MB, which it says defaults to 20MB (in contrast to the project description which says 100)

The plugin also provides commands :UnLarge, :Large and :Large! to disable, reenable, or force enable (for small files) respectively on the currently loaded file.

2

I have heard that turning off syntax highlighting can help:

:syntax off
  • Having any kind of folding enabled also has a massive effect – craigp Feb 4 '15 at 7:38
0

Another pager you may like to try is most: http://www.jedsoft.org/most/

Also, rotate your logs and as this may contribute to less larger logs.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.