2

I've known that Vim 8 provides us an async mechanism job_start so that we can start a command asynchronously and get its return with the help of channel.

However, it seems that job_start can only execute the OS command, such as make, grep, /bin/sh etc.

Is it possible to call a vim script function with the help of job_start?

Here is my vimrc:

function! CustomGrep(...)
    // do something
endfunction
command! -nargs=? Grep :call CustomGrep(<f-args>)

As you see, I can :Grep xxx in my vim to call the function CustomGrep. Now I want to make this async, something like this: command! -nargs=? Grep :job_start(...), but I don't know how to achieve this.

I've also tested like this: command! -nargs=? Grep : call job_start('echo 111'), but I can't see 111 in my vim when I execute Grep xxx.

Here is my real custom grep: https://github.com/YvesZHI/Camel-vim/blob/vim8/Camel.vim. You can read this from the line 240 to the end.

In a word, as I starts to use vim 8, which provides some kind of async-job mechanism, I'm thinking if it's possible to make my custom grep async. For now my custom grep could work as expected. Sometimes it does take a while to complete but normally it's just 2 or 3 seconds, that's not a big deal. But what if I'm working with a very huge project? So it gonna be great if I can make my custom grep async.

6
  • Jobs start processes. A vim function isn't a process though it runs in one (the Vim process). You could invoke another instance of Vim and call a function in that instance or perhaps use a timer to asynchronously call the function in the current instance. See the four item list at the top of channel.txt in help for job uses. – B Layer Aug 26 '20 at 7:36
  • Can you explain more about what you're trying to do. Why do you want to make an async function call? The reason I posted a comment and not an answer is I felt I needed more information before I could make a definitive recommendation. – B Layer Aug 28 '20 at 0:17
  • @BLayer In fact I've used the command :vimgrep to make my custom grep. It worked. Now I starts to use vim 8 so I'm thinking if it's possible to make my custom grep async. – Yves Aug 28 '20 at 1:05
  • @BLayer github.com/YvesZHI/Camel-vim/blob/vim8/Camel.vim You could read this link: from the line 240 to the end. This is my custom grep. – Yves Aug 28 '20 at 1:06
  • @BLayer Well, it's more like an exploratory thing. For now my custom grep does take a while to complete but it's OK for me. As vim 8 supports async job, so I'm simply thinking if I can make some improvement on this. – Yves Aug 28 '20 at 3:45
5

From discussion in comments I learned that OP has a custom vimgrep call and would like to run it asynchronously. The utility of this is pretty clear, especially if you run a lot of searches that take a while: by moving the execution into a separate/background thread you free up Vim's UI so you can continue working without waiting for the result.

As I mentioned in one of my comments above, a job is used to asynchronously manage a process. Calling a function in Vim is at a different level of granularity. It all takes place in a single, already-running process... Vim's! ... so that isn't applicable here.

Still, there is a relatively lightweight way to do what we want and that's by using a "timer". Timers are usually used to do one or both of the following:

  • Start a task after a delay.
  • Do a task repeatedly

A sample call makes this pretty clear

let timerid = timer_start(2000, 'RepeatMePlease', { repeat: 5 })

The timer_start() call will return immediately with the assigned timer's id. Meanwhile, in the background, a delay of 2 seconds will occur after which a thread will be assigned to run the function RepeatMePlease(). The delay and call will be repeated four more times after that.

What is the timer ID for? Well, you can stop, pause, or get bookkeeping info about an existing timer. To do so you obviously need some way to reference it. The timer ID.

(Note: I said "after 2 seconds" but that's actually the minimum delay. If Vim is busy it might be longer than that.)

Anyways, what does that have to do with making a simple background call? Well, if you don't do any repeats and you don't have a delay what's left? A function running in the background.

The one complication here is that, as you may have noticed, there are no parameters specified for RepeatMePlease(). That's because there's nowhere to put them. In fact, when you pass a function name that function will be called by Vim with a single parameter: the timer id.

That's pretty restricting. What ever shall we do?

Fortunately, there's an alternative: we can define a function inline. This is known as a lambda expression in Vim (and elsewhere but sometimes it's called an "anonymous function"). These have a property that is very useful: they act as closures. Huh? All you need to know here is that this allows us to access local variables and parameters. I'm not going to go into much more detail than that. That's what help and google are for. Instead, let's just see it in action. (Don't get scared. There are only three lines of executed code here.)

" This is the entry point for the custom vimgrep. Let's say that you
" just pass in a pattern that was entered by the user.
function! PretendCustomGrep(patt) abort
    " Here you can do whatever you need to prepare the grep call then...

    " I've added a 3 second delay so you can prove to your self
    " that this function returns right away and you can use Vim normally.
    " In reality you don't need this.

    call timer_start(3000, {-> execute("call LongRunningFun('" . a:patt . "')", "")})

endfunc

" This does the actual grep stuff
function! LongRunningFun(patt) abort
    " When you have your result display it, open it, open quickfix, or
    " whatever it is you want to do with it. Here I'm just showing something
    " in a short-lived popup.

    let l:msg = "Search for " . a:patt . " found: A RESULT!"
    call popup_notification(l:msg , #{ line: 5, col:10, highlight: 'Warning', } )

endfunction

Update: You can accomplish the same thing without lambdas and closures by using a Funcref variable. You get a Funcref from the function() function. function() takes a function name and a list. The list contains whatever parameters you need to pass to the function. Yes, that's a little confusing so let's just look at the revised use of timer for the first function above...

    let fref = function('LongRunningFun', [patt]) 
    call timer_start(3000, fref)

or, alternatively, just inline it...

    call timer_start(3000, function('LongRunningFun', [patt]))

There's a little less flexibility this way but it'll be preferable in the vast majority of cases.

2
1

Is it possible to call a vim script function with the help of job_start?

Not directly. I guess you would need Vim to be multithreaded, which might be possible in the future.

In fact I've used the command :vimgrep to make my custom grep. It worked. Now I starts to use vim 8 so I'm thinking if it's possible to make my custom grep async.

FWIW, I wrote some code with the same purpose a long time ago:

com -nargs=* Vim call s:vimgrep(<q-args>)
com -nargs=* Lvim call s:vimgrep(<q-args>, v:true)

fu s:vimgrep(args, loclist = v:false) abort
    let tempqfl = tempname()
    let tempvimrc = tempname()
    let get_tempfile =<< trim END
        let tempqfl = expand('%:p')
        if tempqfl !~# '^/tmp/'
            finish
        endif
    END
    let cdcmd = 'cd ' .. getcwd()->fnameescape()
    let setcmd = printf('set wildignore=%s suffixes=%s %signorecase %ssmartcase',
        \ &wildignore, &suffixes, &ignorecase ? '' : 'no', &smartcase ? '' : 'no')
    let args = s:expandargs(a:args)
    let vimgrepcmd = 'noa vim ' .. args
    let getqfl =<< trim END
        eval getqflist()
           \ ->map({_, v -> printf('%s:%d:%d:%s',
           \     bufname(v.bufnr)->fnamemodify(':p'),
           \     v.lnum,
           \     v.col,
           \     substitute(v.text, '[^[:print:]]', {m -> strtrans(m[0])}, 'g')
           \ )})
           \ ->writefile(tempqfl, 's')
        qa!
    END
    call writefile(['vim9script']
        \ + get_tempfile
        \ + [cdcmd, setcmd, vimgrepcmd]
        \ + getqfl,
        \ tempvimrc, 's')
    let vimcmd = printf('vim -es -Nu NONE -U NONE -i NONE -S %s %s', tempvimrc, tempqfl)
    let title = (a:loclist ? ':Lvim ' : ':Vim ') .. args
    let arglist = [a:loclist, tempqfl, title]
    let opts = #{exit_cb: function('s:callback', arglist)}
    call split(vimcmd)->job_start(opts)
endfu

fu s:callback(loclist, tempqfl, title, _j, _e) abort
    let efm_save = &l:efm
    let bufnr = bufnr('%')
    try
        setl efm=%f:%l:%c:%m
        if a:loclist
            exe 'lgetfile ' .. a:tempqfl
            lw
            call setloclist(0, [], 'a', #{title: a:title})
        else
            exe 'cgetfile ' .. a:tempqfl
            cw
            call setqflist([], 'a', #{title: a:title})
        endif
    finally
        call setbufvar(bufnr, '&efm', efm_save)
    endtry
    redraw!
endfu

fu s:expandargs(args) abort
    let pat = '^\(\i\@!.\)\1\ze[gj]\{,2}\s\+'
    let rep = '/' .. escape(@/, '\/') .. '/'
    return substitute(a:args, pat, rep, '')
        \ ->substitute('\s\+\zs%\s*$', expand('%:p')->fnameescape(), '')
        \ ->substitute('\s\+\zs##\s*$', argv()
        \ ->map({_, v -> fnamemodify(v, ':p')->fnameescape()})
        \ ->join(), '')
endfu

The code installs 2 custom commands :Vim and :Lvim which are some kind of asynchronous wrappers around the builtin :vimgrep and :lvimgrep. As an example, you can test it by running this command:

:Vim /pattern/gj $VIMRUNTIME/**

This should asynchronously populate a quickfix list with all the lines where pattern is written in any file in your $VIMRUNTIME directory.

It seems to work, but I don't use it regularly, so I don't know how reliable it is.


Re-written in Vim9 script:

vim9script

com -nargs=* Vim Vimgrep(<q-args>)
com -nargs=* Lvim Vimgrep(<q-args>, true)

def Vimgrep(args: string, loclist = false)
    var tempvimrc: string = tempname()
    var tempqfl: string = tempname()
    var get_tempfile: list<string> =<< trim END
        var tempqfl = expand('%:p')
        if tempqfl !~ '^/tmp/'
            finish
        endif
    END
    var cdcmd: string = 'cd ' .. getcwd()->fnameescape()
    var setcmd: string = printf('set wildignore=%s suffixes=%s %signorecase %ssmartcase',
        &wildignore, &suffixes, &ignorecase ? '' : 'no', &smartcase ? '' : 'no')
    var _args: string = Expandargs(args)
    var vimgrepcmd: string = 'noa vim ' .. _args
    var getqfl: list<string> =<< trim END
        getqflist()
           ->mapnew((_, v: dict<any>): string => printf('%s:%d:%d:%s',
               bufname(v.bufnr)->fnamemodify(':p'),
               v.lnum,
               v.col,
               substitute(v.text, '[^[:print:]]', (m: list<string>): string => strtrans(m[0]), 'g')
               ))
           ->writefile(tempqfl, 's')
        qa!
    END
    writefile(['vim9script']
        + get_tempfile
        + [cdcmd, setcmd, vimgrepcmd]
        + getqfl,
        tempvimrc, 's')
    var vimcmd: string = printf('vim -es -Nu NONE -U NONE -i NONE -S %s %s', tempvimrc, tempqfl)
    var title: string = (loclist ? ':Lvim ' : ':Vim ') .. _args
    var arglist: list<any> = [loclist, tempqfl, title]
    var opts: dict<func> = {exit_cb: function(Callback, arglist)}
    split(vimcmd)->job_start(opts)
enddef

def Callback(loclist: bool, tempqfl: string, title: string, _j: job, _e: number)
    var efm_save: string = &l:efm
    var bufnr: number = bufnr('%')
    try
        setl efm=%f:%l:%c:%m
        if loclist
            exe 'lgetfile ' .. tempqfl
            lw
            setloclist(0, [], 'a', {title: title})
        else
            exe 'cgetfile ' .. tempqfl
            cw
            setqflist([], 'a', {title: title})
        endif
    finally
        setbufvar(bufnr, '&efm', efm_save)
    endtry
    redraw!
enddef

def Expandargs(args: string): string
    var pat: string = '^\(\i\@!.\)\1\ze[gj]\{,2}\s\+'
    var rep: string = '/' .. escape(@/, '\/') .. '/'
    return substitute(args, pat, rep, '')
        ->substitute('\s\+\zs%\s*$', expand('%:p')->fnameescape(), '')
        ->substitute('\s\+\zs##\s*$', argv()
        ->map((_, v: string): string => fnamemodify(v, ':p')->fnameescape())
        ->join(), '')
enddef

This requires a recent Vim version. It works on 8.2.2332.

2
  • 1
    Why set tempqfl to tempname() and the override it with expand? And why the guard? – D. Ben Knoble Aug 29 '20 at 13:13
  • 1
    It's just a sanity check. The second Vim instance should open a temporary file whose name has been generated in the first Vim instance. It should be used to write the quickfix list data structure, so that the callback in the first Vim instance can retrieve it. After a refactoring, I remember overwriting a valuable file by accident. If the logic changes again, I would not want losing valuable info again. – user938271 Aug 29 '20 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.