0

I have a typical python project structure:

├── requirements.txt
└── src.py

where requirements.txt is holding the requirements of my project, and src.py has my code.

In my minimal example, they look like this:

# requirements.txt

numpy==1.21.3
# src.py

import numpy

a = numpy.array([])

When I edit this in nvim, with some LanguageServer (pyright in my case) functionality, numpy cannot be resolved - as it is not installed.

I know I can make this happen by using a virtual environment. But I am looking for a solution without needing to activate a virtual environment, as it is not part of my workflow (I run everything in docker containers.)

Is there another way to make the LanguageServer know the dependencies per project? (I would be willing to define the files holding the requirements per project, of course, but they also follow the simple pattern of being either located in a requirements/ directory or simply be named `requirements.txt.)

I understand, this is technically not a question about vim, but about the LanguageServer. So, if this is out of scope here, feel free to close this. I just thought, the vim.SE would be best guess when looking for people who are familiar with LSP.

3
  • 1
    That's an interesting question but conceptually I'm not sure how that would work: Your LSP server needs to have the declarations stored somewhere, if you don't install the dependencies how is LSP supposed to find these declaration? Looking them up online would make the suggestions super slow. So it would need to lookup the declarations somewhere, store them in a way or another and then used these stored declaration which is just the equivalent of installing your dependencies, right?
    – statox
    Oct 21 at 9:10
  • @statox Yes, you are right. I think, it won't be possible without installing the dependencies. I would like to avoid at least, to activate the virtualenv every time I edit code in the project. I just found out, that pyright allows to store a pyrightconfig.json, where I can just place the path to the virtualenv. I think, I am going to it like that. So, I will have the dependencies installed somewhere on my machine, and link them per project with pyrightconfig.json Oct 21 at 11:39
  • That sounds like a reasonable option. Don't hesitate to self answer your question if you find a setup that works for you, that might be useful for other people :)
    – statox
    Oct 21 at 12:26
1

As statox pointed out in a comment, there is probably no way around having the dependencies installed somewhere on the machine:

Your LSP server needs to have the declarations stored somewhere, if you don't install the dependencies how is LSP supposed to find these declaration? Looking them up online would make the suggestions super slow. So it would need to lookup the declarations somewhere, store them in a way or another and then used these stored declaration which is just the equivalent of installing your dependencies, right?

Regarding the special case of pyright, I found that it is possible to tell pyright in a pyrightconfig.json which virtualenv it should use. That way, I can at least avoid having to activate the virtual environment every time I edit the code.

So, I have written a small python script in order to create the virtual environment from anywhere in the project. It uses a heuristic to find the root directory of the project and install put the pyrightconfig.json there, based on the existence of some files and subdirectories (Makefile and docker-compose.yml, in my case). It also installs the virtualenv in the project root. (This location could easily be changed to a central directory machine.)

Just in case someone could use it, this is the script:

from json import dump
from os import getcwd, system, walk
from os.path import dirname, exists, isfile, join as pathjoin
from typing import List, Tuple

VENV = 'venv'


def execute(*commands: str) -> None:
    command = ' && '.join(commands)
    system(command)


def has_file(path: str, filename: str) -> bool:
    filepath = pathjoin(path, filename)
    return isfile(filepath)


def get_project_root(path: str) -> str:
    IDENTIFIERS = ['Makefile', 'docker-compose.yml']
    if all(has_file(path, identifier) for identifier in IDENTIFIERS):
        return path
    else:
        return get_project_root(dirname(path))


def get_requirements_filepaths(path: str) -> List[str]:
    IDENTIFIERS = [
        'requirements.txt',
        pathjoin('requirements', 'dev.txt'),
        pathjoin('requirements', 'common.txt'),
    ]
    for root, dirs, _ in walk(path):
        for name in dirs:
            results = []
            for identifier in IDENTIFIERS:
                filepath = pathjoin(root, name, identifier)
                if isfile(filepath):
                    results.append(filepath)
            if results:
                return results
    return []


def make_venv(path: str) -> Tuple[str, bool]:
    venv_path = pathjoin(path, VENV)
    venv_exists = exists(venv_path)
    if not venv_exists:
        execute(
            f'cd {path}',
            f'virtualenv {VENV} --python=python3',
        )
    return venv_path, venv_exists


def install_requirements(
    venv_path: str,
    requirements_filepaths: List[str],
) -> None:
    install_commands = [
        f'pip install -r {filepath}'
        for filepath in requirements_filepaths
    ]
    execute(
        f'. {venv_path}/bin/activate',
        *install_commands,
        'deactivate'
    )

def write_pyrightconfig(dir: str) -> None:
    path = pathjoin(dir, 'pyrightconfig.json')
    config = {
        'venvPath': dir,
        'venv': VENV,
    }
    with open(path, 'w+', encoding='utf8') as configfile:
        dump(config, configfile, indent=4)


if __name__ == '__main__':
    cwd = getcwd()
    project_root = get_project_root(cwd)
    venv_path, venv_existed = make_venv(project_root)
    if not venv_existed:
        requirements_filepaths = get_requirements_filepaths(project_root)
        install_requirements(venv_path, requirements_filepaths)
    write_pyrightconfig(project_root)

I have put a short alias in my .bashrc, which I can run from anywhere in a project, to trigger the script.

So, with this solution, I still have to maintain the dependencies in the virtualenv to be up to date, but this is a non-reducible problem, as it is required to have the dependencies installed somewhere on my machine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.