I have the following .vimrc:

let &makeprg="./run-tests"

The contents of run-tests:


coverage run --source "$PWD" --branch -m pytest tests/ >"$error_file" 2>&1
if test $?; then
  coverage report --show-missing
  cat "$error_file"

If the tests pass, I want to only show the coverage information. If the tests fail, I only want to show the test information.

However, on both passing and failing tests, $? is always 0, i.e. a success. If I copy and paste the coverage run ... command in the terminal and manually inspect echo $? I do get 0 for success and 1 for failing tests. It seems that the value of $? is being altered by some other command that Vim is executing in the makeprg pipeline.

How best to resolve this?

  • Your script would run in a separate process (a subshell) and use of $? within won't be influenced by whatever happens external to the script. Instead of running coverage run ... directly/manually you should try running the script itself. Maybe save $? in a variable immediately after coverage (e.g. result=$?) then, say, echo "Result is $result" before doing if test $result; then .... Then at least you'll know for sure what it's value was in the script.
    – B Layer
    Commented Aug 17, 2021 at 9:10
  • 1
    It’s a bit of an anti pattern to just test the exit code this way, when you can do if command …. @BLayer is correct: vim cannot interfere with the execution of the script as you describe. Perhaps you misunderstood the test builtin? Does the script work outside of vim? I suspect this belongs somewhere more related to programming than here.
    – D. Ben Knoble
    Commented Aug 17, 2021 at 11:33

1 Answer 1


The problem you're having here is that the test command with a single argument will by default check whether or not that's an empty string. So, in your case, it's taking the 0 and looking at it as a "0" string, which is not empty (it has one character), but the same with a fail error code, like 1, which it sees as "1" and it's also non-empty.

For your code to work and properly check whether the coverage run succeeded, what you need here is test $? = 0 instead.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.