1

EDIT2: A user suggested that I may be going about this in a fundamentally wrong way, in particular in that I am not using remote (new-style) plugins and am trying to use async python code where it might not belong. As such, I would like to briefly reframe the question here and then leave the original text below.

I am a novice attempting to develop a neovim plugin in python3 that runs some long-blocking operations on startup and then reports a result. I would like to do this asynchronously for obvious reasons. Can anyone provide a concrete, minimal example of how to do that that can get me on my feet and coding in the right direction, assuming no particular knowledge of vim plugin development?

ORIGINAL TEXT:

I have a long-running (specifically, long-blocking) plugin written in python that runs once at startup and reports its result when finished. Let's imagine it as

import vim
import time

def plugin_sync():
    time.sleep(5)
    print("done")

vim.async_call(plugin_sync)

I had hoped this wouldn't impact startup, and indeed it doesn't, if run in isolation.

However, if I add certain other plugins, say using vim-plug and an init vim with lots of work to do like

call plug#begin('~/.local/share/nvim/plugged')
Plug 'me/my-long-running-plugin'
Plug 'neoclide/coc.nvim', {'branch': 'release'}
call plug#end()

let g:coc_global_extensions=["coc-json", "coc-eslint", "coc-tsserver",
  \ "coc-snippets", "coc-html", "coc-css", "coc-python", "coc-java",
  \ "coc-highlight", "coc-yank", "coc-omnisharp", "coc-emmet",
  \ "coc-lists", "coc-marketplace", "coc-neosnippet", "coc-prettier",
  \ "coc-clangd", "coc-cmake", "coc-xml" ]

, nvim becomes unresponsive for 5 seconds. Per the docs,

Note that this code will still block the plugin host if it does long-running computations. Intensive computations should be done in a separate thread (or process), and vim.async_call can be used to send results back to Neovim.

, so I'm guessing that the unresponsiveness has to do with another plugin blocking nvim while waiting for the plugin host.

Briefly ignoring the advice in the docs (I'll get back to that), it seemed like maybe the right thing to do was just use proper async python as follows:

import vim
import asyncio

async def plugin_async():
    await asyncio.sleep(5)
    print("done")

vim.async_call(lambda: asyncio.create_task(plugin_async()))

This doesn't block, but it also doesn't print messages. I know that it's running, because I can insert some side effect like touching a file, but I would also like to be able to print a message after finishing, which is really the whole point of the plugin.

My questions, in order of specificity, are as follows:

  1. How can I print messages or otherwise deliver feedback to the user from python code running in the asyncio event loop?
  2. Is what I've done "valid," in the sense of not having any weird consequences I may not have imagined?
  3. Circling back to the docs, what is the "standard" way to do this? I've seen suggestions like use neomake or vim-dispatch, or use job control, but I'm very novice with vim script, and I can't say I particularly want to learn a bunch of new concepts to accomplish something that it looks like I can probably just do in pure python. That said, if my way is infeasible or otherwise "wrong," it would be really helpful if responders could give some concrete examples of how to do e.g. what the docs suggest but don't elaborate on.

EDIT: I noticed that vim.eval doesn't work properly inside of async functions:

import vim
import asyncio
import warnings

async def plugin_async():
    try:
        vim.eval("g:defined_var")
    except Exception as e:
        warnings.warn(e)

vim.async_call(lambda: asyncio.create_task(plugin_async()))

Note that I'm using warnings for convenience because they are communicated back to the user, whereas uncaught exceptions are suppressed. This produces

UserWarning: 'NoneType' object has no attribute 'switch'

, which is as yet a mystery to me. The same code works just fine in a sync function, so there's something else going on here that I don't understand. Help following what's going on would be much appreciated.

3
  • 1
    Is this a rplugin/python3/ plug-in? In short, you should call vim.async_call() only for the parts that don't take a long time... In your example, you should not call it for the time.sleep(5) but then call it for the print("done"), for example, vim.async_call(print, "done"). Your use of Python async def and asyncio is actually mudding the waters even more... Note that vim.async_call() has nothing to do with Async Python. So it would be much easier if you didn't use that in your example... Please edit and add more details about rplugin/ and I'll try to write an answer.
    – filbranden
    Jul 24, 2021 at 1:20
  • 1
    @filbranden, I think I am probably not using the most modern approach. It's been a while since I originally attempted this, but I believe I followed tinyurl.com/4v55umh2, which outlines an approach to setting up the path and then calling some python from a vim script which itself imports a python script. I'm guessing things have evolved considerably since then, assuming it was ever even a good approach... So I have no rplugin/ details to add, but am very open to that approach if you would address long-blocking calls in that style Jul 24, 2021 at 19:21
  • 1
    @filbranden, re: usage of vim.async_call and "Intensive computations... separate... process... vim.async_call... send results back to Neovim," is the pattern to e.g. start a process (but not block to join, or maybe register my process handle somehow to be gathered later?), and then call vim.async_call in the forked (or OS equiv) process to send information back? I tried that just now without success, though it does seem like a plausible interpretation where I might just be missing some details. Apologies if way off base; I'm in a real I don't know what I don't know situation Jul 24, 2021 at 22:32

1 Answer 1

1

Although this question raises some as yet unanswered questions about how to best handle doing long-running stuff in the traditional :python style plugin, it seems mostly moot now that I've spent some time understanding the new style remote plugin system. nvim-example-python-plugin was helpful (though slightly outdated), so I'd recommend anyone else finding themselves floundering start there.

The key observation is this: remote plugins are already running in their own processes (hence remote), and the vim object is used for RPC. As such, it doesn't matter at all if the plugin blocks, so long as it doesn't instruct the plugin manager to do any blocking work, which was of course exactly what I was doing using vim.async_call to do blocking work.

As for python async code, there's a line at the end of Nvim.__init__ that exposes the event loop for python 3, so if one wanted to run async code (python >= 3.5), one can schedule tasks using vim.loop.

Here's a minimal example:

import pynvim
from pynvim.api.nvim import Nvim
from time import sleep
import asyncio


@pynvim.plugin
class MyPlugin(object):
    def __init__(self, nvim: Nvim) -> None:
        self.nvim = nvim

    @pynvim.function("LongBlock")
    def long_block(self, args):
        sleep(5)
        self.nvim.command('echo "done with blocking stuff"')

    @pynvim.function("AsyncNoBlock")
    def async_no_block(self, args):
        async def no_block():
            await asyncio.sleep(5)
            self.nvim.command('echo "done with async stuff"')

        self.nvim.loop.create_task(no_block())

Weirdly, vim.async_call(print, "message") doesn't echo, which I thought it would based on @filbranden's comment. I'm going to keep this unanswered until that is resolved, and perhaps until more experienced others can point out how I'm still probably missing some other things.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.