Swapping between files causes unwanted insertion of newline by vim

Question

I have a blank .vimrc file. I'm seeing some very strange behavior. Here's what happens:

1. I open a file in vim
2. I do :split .tmp/file and :q
3. I make a change to the open file and :w
4. I run :e # which brings up the alternate file opened in step 2 and then swap back to first file with :e # without doing anything else.
5. I run gx, a command from plugin I wrote (see below) that gets input from the user from vim's status bar which then passes it to a perl script. The perl script appends and writes two lines and the message to the .tmp/file.
6. THE PROBLEM: When I reopen the .tmp/file, there are three newlines after the last message instead of two. So somehow, vim is slapping an extra newline character into my .tmp/file.

Note: This problem does not happen when I do not run steps 2, 3, and 4; everything works as expected with two lines appended after each message. Here is my plugin (yeah, it's poorly written, it's my first one):

nnoremap gx  :call AddCommNote()<CR>

let g:Msg = ''

function! AddCommNote()
  call inputsave()
  let g:Msg = input('Make commit message: ')
  call inputrestore()
  let g:Msg = call substitute(g:Msg, '\n', '', '')
  let is_commit = 0
  call ProcessMsg(is_commit)
endfunction

function! ProcessMsg(is_commit)
  let escaped = substitute(g:Msg, "'", "'\"'\"'", "")
  let command_to_run = "add_comm_note '" . escaped . "' "
  let command_to_run = command_to_run . "'" . expand("%:p") . "' " . a:is_commit
  exec "silent:!" . command_to_run
  :redraw!
endfunction

I've stripped all newlines from g:Msg in the plugin and in the perl script. The newline is definitely coming from vim as far as I can tell. I have no idea how.

Here is the simple perl script which is in my ~/bin directory:

#!/usr/bin/perl

use strict;
use warnings;
use File::Util;

my $f = File::Util->new();
my $message = $ARGV[0];
$message =~ s/\n//g;
my $file = $ARGV[1];
my ($path) = $f->return_path($file);
my ($name) = $f->strip_path($file);
my $tmp_file = $path . '/.tmp/' . $name;
my $newlines = $f->existent($tmp_file) ? "\n\n" : '';

if ($ARGV[2] && !$message && !$newlines) {
   $message = 'this commit message is unhelpful';
}

exit if (!$newlines && !$message); #nothing to add to the commit message file

$f->write_file('file' => $tmp_file, 'mode' => 'append', 'content' => $newlines . $message);

Answer 1

I know nothing about perl, so if the problem comes from the script, or an interaction between Vim and the script I can't help you.

That being said, for the Vim part, here's how I would do it:

nno gx  :<C-U>call AddCommNote()<CR>

let g:Msg = ''

fu! AddCommNote() abort
    let g:Msg = substitute(input('Make commit message: '), '\n', '', 'g')
    let is_commit = 0
    call ProcessMsg(is_commit)
endfu

fu! ProcessMsg(is_commit) abort
    echom printf('silent !add_comm_note %s %s %d',
                \   shellescape(g:Msg, 1),
                \   shellescape(expand('%:p'), 1),
                \   a:is_commit
                \ )
    redraw!
endfu

---

There are 2 main differences between this version and yours.

First, it adds the g flag to the substitute() function, so that all newlines are stripped and not only the first one, and it removes the :call token from the assignment (:call is only needed when you want to run the code inside a function; it's an error if you use it inside an assignment). So, these lines:

let g:Msg = input('Make commit message: ')
let g:Msg = call substitute(g:Msg, '\n', '', '')

… could be replaced with:

let g:Msg = substitute(input('Make commit message: '), '\n', '', 'g')

The second difference is the use of shellescape() and printf(). This part:

let escaped = substitute(g:Msg, "'", "'\"'\"'", "")
let command_to_run = "add_comm_note '" . escaped . "' "
let command_to_run = command_to_run . "'" . expand("%:p") . "' " . a:is_commit
exec "silent:!" . command_to_run

… could be replaced with:

  exe printf('silent !add_comm_note %s %s %d',
          \   shellescape(g:Msg, 1),
          \   shellescape(expand('%:p'), 1),
          \   a:is_commit
          \ )
  redraw!

:execute should execute the output of the printf() function.

printf() expects a string describing a template containing various items. Each item inside the template is preceded by a % sign. Here, the template passed to printf() as a first argument is:

silent !add_comm_note %s %s %d

It contains 3 items %s, %s, %d. Each item will be replaced with the next arguments passed to printf(). %s denotes a string, while %d denotes a decimal number. The next arguments passed to printf() are:

shellescape(g:Msg, 1)            the value of this expression will replace the first `%s`
shellescape(expand('%:p'), 1)    the value of this expression will replace the second `%s`
a:is_commit                      the value of this expression will replace the last `%d`

The shell arguments that you pass to your perl script, could have special characters which have a special meaning to the shell or to Vim. To protect them, shellescape() is called (protection from the shell), and 1 is passed as a second argument (protection from Vim). See :h shellescape() for more info.

---

If this new version doesn't work, you could replace :exe with :echom:

fu! ProcessMsg(is_commit) abort
  echom printf('silent !add_comm_note %s %s %d',
           \    shellescape(g:Msg, 1),
           \    shellescape(expand('%:p'), 1),
           \    a:is_commit
           \ )
  redraw!
endfu

After typing your mapping, gx, the whole command should be logged inside the message-history (see :h :echom). You could read it by typing :messages, then hit G to go to the bottom of the history. Maybe you could read what it looks like, and make some tweaks according to what you want the shell to execute.

---

If it doesn't help because the problem comes from an interaction between Vim and the perl script, I'll delete the answer. I just wrote this in case it have an effect on the issue.

---

Edit:

To better understand where the bug comes from, you could also use the system() function to capture the output of the shell command inside a global variable (e.g. g:debug):

fu! ProcessMsg(is_commit) abort
    let g:debug = system(printf('add_comm_note %s %s %d',
                \   shellescape(g:Msg, 1),
                \   shellescape(expand('%:p'), 1),
                \   a:is_commit
                \ ))
endfu

Then, read the output with :echo g:debug (and maybe post it here in your question, if you think it's relevant to the issue).