How does ft-c-omni work and how can I make it faster?

Question

As I was studying the various ways of doing completion in Vim 8.0, I came across the very interesting "omni-completion" feature and its 'omnifunc' option, which is specialized as ccomplete#Complete for the C filetype (:h ft-c-omni).

I tried it out on a "Hello, World!" example and I was pleased by the promising results so I decided to put it to the test by trying to complete the field of a structure somewhere in the Linux Kernel (after having run make tags). On a fairly decent laptop, this took over a minute, which is slightly problematic for anyone with a speed of typing above 1 WPM.

So how is that feature implemented and is there an undocumented way of making it go faster (pre-computing, caching, ...) so that it can be usable in large projects such as the Kernel? Or is ft-c-omni effectively useless for such real-world use-cases?

Answer 1

So how is that feature implemented and is there an undocumented way of making it go faster (pre-computing, caching, ...) so that it can be usable in large projects such as the Kernel? Or is ft-c-omni effectively useless for such real-world use-cases?

The C omni completion function is implemented in $VIMRUNTIME/autoload/ccomplete.vim.

I profiled it with a very simple test:

$ vim -S <(cat <<'EOF'
    cd ~/Vcs/vim
    e src/evalfunc.c
    prof start /tmp/profile.log
    prof func ccomplete#Complete
    so $VIMRUNTIME/autoload/ccomplete.vim
    au VimEnter * call feedkeys("Oe\<c-x>\<c-o>", 'xt') | qa!
EOF
)

Here are the results.

Your environment is different, so you probably want to profile the function on your machine. Here's how:

• run :prof start /tmp/profile.log to tell Vim you want to profile "something", and that the results should be written inside /tmp/profile.log
• run :so $VIMRUNTIME/autoload/ccomplete.vim to tell Vim to source the script where the ccomplete#Complete() function is defined
• run :prof func ccomplete#Complete to tell Vim that the "something" you want to profile is the function ccomplete#Complete()

Now, all you have to do is to call the function at least once (i.e. press C-x C-o after some text inside a file of your project), then quit Vim. See :h profiling.

On my machine, the function spends little time in executing code of its own; most of the time is spent in code run by nested function calls:

FUNCTIONS SORTED ON TOTAL TIME
count  total (s)   self (s)  function
    2   0.775950   0.065477  ccomplete#Complete()
                   ^------^
                   only 8% of the total time is spent for its own code

Now you need to find out which nested function calls take the most time; on my machine there are two:

1   0.245847   0.006270     call extend(res, map(tags, 's:Tag2item(v:val)'))
1   0.476847   0.005951   return map(res, 's:Tagline2item(v:val, brackets)')
│   │          │
│   │          └ but it took only that much time for the command itself
│   └ it took that much time in total
└ the command has been executed only once

They call s:Tag2item() and s:Tagline2item(). I've profiled s:Tagline2item() which is the slowest of them: here are the results.

Again, you probably want to profile the function on your machine. The procedure is exactly the same as before; with one exception; you can't run this:

:prof func s:Tagline2item
           ^^
           ✘

You must translate s: into a sequence such as <SNR>123_.

:prof func <SNR>123_Tagline2item
           ^-------^
               ✔

Where 123 is the id of the script where s:Tagline2item() is defined. You can get it by running :scriptnames, or you can type this on the command-line:

:func s:Tagline2item

Then:

• press Tab to make Vim translate s:
• press C-b to jump at the start of the command-line
• insert prof
• press Enter to run the command

On my machine, s:Tagline2item() spends little time in executing code of its own; most of the time is spent in code run by nested function calls:

FUNCTIONS SORTED ON TOTAL TIME
count  total (s)   self (s)  function
 1036   0.483790   0.057566  <SNR>178_Tagline2item()
                   ^------^
                   only 12% of the total time is spent for its own code

Again, you need to find out which command(s) take(s) the most time; on my machine it's this one:

1036   0.436545   0.010321   let add = s:GetAddition(line, a:val['match'], [a:val], a:brackets == '')

It calls s:GetAddition() (here is its profile), which is slow mainly because of this command:

757   0.401642   0.007140   if len(s:SearchMembers(a:memarg, [''], 0)) > 0

It calls s:SearchMembers() (here is its profile), which is slow mainly because of this command:

33   0.299922   0.000560      call extend(res, s:Nextitem(strpart(line, s, e - s), a:items, 0, a:all))

It calls s:Nextitem() (here is its profile), which is slow mainly because of this command:

33              0.281755     let diclist = taglist('^' . tokens[tidx] . '$')

This command alone takes more than a third of the total time used by C-x C-o.

To make it faster you could try to cache the output of taglist(), and only invoke it if the necessary information can't be found in the cache.

I didn't profile every function to find all the slowest commands, but there are probably a few others that you could refactor to make the omni completion faster. If you go that route, don't perform your experiments directly in $VIMRUNTIME; do them in your ~/.vim. Write your autoload script in ~/.vim/autoload/ccomplete.vim:

:e ~/.vim/autoload/ccomplete.vim
:r $VIMRUNTIME/autoload/ccomplete.vim

Since ~/.vim comes before $VIMRUNTIME in the runtimepath, when C-x C-o will invoke ccomplete#Complete(), your definition will be found first; the other one will be ignored.

---

You could also try to refactor the code in Vim9 script. See here for an example.

It works on 8.2.2332. With this implementation, the commands won't be parsed every time you press C-x C-o; they will be compiled into instructions which can be executed more efficiently. You can read them by executing:

:pu=execute('disassemble ccomplete#Complete')

It should write a long list of instructions, such as:

ccomplete#Complete
    if findstart
   0 LOAD arg[-2]
   1 JUMP_IF_FALSE -> 146


        let line = getline('.')
   2 PUSHS "."
   3 BCALL getline(argc 1)
   4 STORE $0


        let start = col('.') - 1
   5 PUSHS "."
   6 BCALL col(argc 1)
   7 PUSHNR 1
   8 OPNR -
   9 STORE $1
   ...

Note that since ccomplete#Complete() is autloaded, and :disassemble only works on installed functions, you must have pressed C-x C-o at least once.

I doubt it will help that much, because the main bottleneck is probably not Vim script here, but it's worth a try.