From a TextChanged autocommand event, how to get the range that has been changed?

Question

Problem

I am trying to implement "trimming trailing whitespace in changed lines only". For this purpose, I intercept TextChanged and TextChangedI autocmd events and record the current line (line(.)) in the handler. The changed lines are then trimmed in a BufWritePre autocmd event handler.

It works OK, except for the case when a line is inserted/removed. In this case, I need to shift existing records.

Initially I assumed that the cursor always gets placed on last inserted/first removed line (with this knowledge, it is possible to calculate the range, if you remember the previous line count), but with undo and paste this is not true.

Question

How to determine the exact line range which has been inserted/removed with current TextChanged event?

Code

The current (broken) python handler code is here (I use amoffat/snake to script in python):

@on_autocmd("TextChanged,TextChangedI", "*")
def trim_whitespace_remember_line(context):
    line_count_old = int(my_get("line_count"))
    line_count = int(vim.eval("line(\"$\")"))
    line = int(vim.eval("line(\".\")"))

    if line_count_old != line_count:
        log("trim_whitespace_remember_line: line_count %d -> %d" % (line_count_old, line_count))
        if line_count > line_count_old:
            # FIXME: we assume that the cursor is placed on the last inserted line
            diff = line_count - line_count_old
            log("trim_whitespace_remember_line: %d lines inserted, current line %d" % (diff, line))
            trim_whitespace_lines_inserted(context, line - diff + 1, line)
        else:
            # FIXME: we assume that the cursor is placed on the first line after removed block,
            # i. e. on the first removed line (before reaccounting)
            diff = line_count_old - line_count
            log("trim_whitespace_remember_line: %d lines removed, current line %d" % (diff, line))
            trim_whitespace_lines_removed(context, line, line + diff - 1)
    else:
        log("trim_whitespace_remember_line: line changed, current line %d" % line)
        lines = my_get("modified_lines")
        line = str(line) + " "
        if lines.rfind(" " + line) == -1:
            lines = lines + line
            my_let(context, "modified_lines", lines)

    my_let(context, "line_count", line_count)

    log("trim_whitespace_remember_line: new lines = '%s'" % my_get("modified_lines"))

Clarification

This all is needed because I often work on projects with a legacy codebase where no one paid attention to trailing whitespace and/or sanity of code formatting altogether.

Trimming all trailing whitespace produces a diff polluted with unrelated changes all over the file, which makes troubles at code review. Hence, I want to avoid trimming whitespace on unchanged lines, but at the same time ensure that there is no trailing whitespace introduced by me.

Answer 1

Is your concern about performance? Calling out to Python code will add unneeded overhead that would make an attempt at performance optimization moot. In any case, you can get the boundaries with getpos() using the marks '[ and ']

let start = getpos("'[")[1:2]
let end = getpos("']")[1:2]

However, I think you may be over-engineering the problem. The solution can be much simpler using autocmd's depending on when you want the trailing whitespace trimmed:

After leaving insert mode

autocmd InsertLeave * '[,']s/\s\+$//e | normal! `^

Using the marks mentioned above, replace trailing whitespace only on the last inserted lines. The /e flag ignores the "pattern not found error". normal! `^ returns your cursor to the last insert position.

On TextChanged

autocmd TextChanged * silent! '[,']s/\s\+$//e | normal! `.

The same as above, but with silent! since the marks may be invalid after a delete/undo. normal! `. returns the cursor to the last change position.

When the buffer is saved

autocmd BufWritePre * g/\s\+$/s///e
autocmd BufWritePost * normal! `^

This is the one I prefer since I don't like surprises while I'm typing. It uses :global to first find lines that have trailing whitespace. Then the s///e substitution reuses the pattern from :global to remove the trailing whitespace. The BufWritePost autocmd puts your cursor back after the buffer is saved.

When the buffer is saved (A better implementation)

function! s:strip_whitespace()
    let saved = @/
    let l = line(".")
    let c = col(".")
    %s/\s\+$//e
    let @/ = saved
    call cursor(l, c)
endfunction

autocmd BufWritePre * call s:strip_whitespace()

This one is wrapped in a function that restores the previous search and cursor position. It uses :%s which operates on all buffer lines. g/\s\+$/s///e already does this, but the pattern would match twice on each line.

Remove blank lines

In case you want to also remove blank lines, you can again use :global to find and remove them, e.g. '[,']g/^\s*$/normal! "_dd

Answer 2

In case anyone is interested, the full code is here (GitHub), and autocmd handler looks like this (again, python code using snake):

@on_autocmd("TextChanged,TextChangedI", "*")
def trim_whitespace_remember_line(context):
    line_count_old = int(my_get("line_count"))
    line_count = int(vim.eval("line(\"$\")"))

    if line_count_old != line_count:
        change_start = int(vim.eval("line(\"'[\")"))
        change_end = int(vim.eval("line(\"']\")"))

        if line_count > line_count_old:
            trim_whitespace_lines_inserted(context, change_start, change_end)
        else:
            diff = line_count_old - line_count
            assert change_start == change_end
            trim_whitespace_lines_removed(context, change_start, change_start + diff - 1)
    else:
        line = vim.eval("line(\".\")") + " "
        lines = my_get("modified_lines")

        if lines.rfind(" " + line) == -1:
            lines = lines + line
            my_let(context, "modified_lines", lines)

    my_let(context, "line_count", line_count)

Answer 3

After reading your clarification, I feel that my answer isn't very correct in the context of your question. I have been in situations where I wished the whitespace wasn't stripped from previously committed files, so I can now appreciate what you are trying to do. It inspired me to write a plugin: wstrip-changed.vim

Here's the script:

" Group a consecutive sequence of lines
function! s:line_groups(lines) abort
  if empty(a:lines)
    return []
  endif

  let start_line = a:lines[0]
  let prev_line = a:lines[0]
  let groups = []

  for l in a:lines
    if l == prev_line || l - 1 == prev_line
      let prev_line = l
    else
      call add(groups, [start_line, prev_line])
      let start_line = l
      let prev_line = l
    endif
  endfor

  call add(groups, [start_line, prev_line])
  return groups
endfunction


function! wstrip_changed#clean() abort
  let bfile = expand('%')
  if empty(bfile) || !filereadable(bfile)
    let lines = range(1, line('$'))
  else
    let lines = systemlist("diff --unchanged-line-format='' --old-line-format='' "
          \ . "--new-line-format='%dn\n' '".bfile."' -", getline(1, '$') + [''])
  endif

  if empty(lines)
    return
  endif

  let view = winsaveview()
  for group in s:line_groups(lines)
    execute join(group, ',').'s/\s\+$//e'
  endfor
  call histdel('search', -1)
  let @/ = histget('search', -1)
  call winrestview(view)
endfunction

The most significant part of the script runs a diff of the buffer content against the existing file and only prints the changed line numbers:

let lines = systemlist("diff --unchanged-line-format='' --old-line-format='' "
      \ . "--new-line-format='%dn\n' '".bfile."' -", getline(1, '$') + [''])

With this, you can easily get the line numbers of the changes without needing complicated change tracking. This can be combined with a single autocmd to strip the whitespace on save:

autocmd BufWritePre *.py StripChangedWhitespace

The script works well enough, but the more rational part of me thinks that removing whitespace from the entire script via %s/\s\+$//e is still preferable.

If you don't own the code, perhaps you can get permission to do a full sweep with sed to strip trailing whitespace from project scripts and make it a single cleanup commit. Then you and your peers won't have to worry about it any more.