How to run a python command based on a matched group of a pattern?

Question

Let say I want to run a command that uses a matched group of a pattern for each line matching the pattern.
The natural choice is :g. With the command :s, there is no problem. If i would do

:g/aaa\(.*\)/s//X\1X

for example, then it would work. But if I would try

:g/aaa\(.*\)/echo \1

It won't work.

I wonder why, and what are the alternatives. I mainly want to run python code, something like:

:g/aaa\(.*\)/python3 print("\1"*3)

Answer 1

TL;DR

There is no simple way to get capture groups into other commands. They simply don't see them.

About the :s command

As many vim users know, :s is the :substitute command, and it deals in vim's flavor of regular expressions.

As many "frequent flyers" know, :s can re-use previous patterns by not providing them. This has lead me to a workflow wherein I will search (/), perfect my pattern, and then "replace" with :%s//<replace>/g--I'm re-using the pattern I searched for.

Global substitute

:g/pattern/s//replacement is pretty much equivalent to :%s/pattern/replacement/--by default, :s only performs substitutions on lines where pattern matches. By using :g instead, you're performing the narrowing of lines first and then invoking the replace. But either way, it accomplishes the same goal.

From the help docs:

The global command sets both the last used search pattern and the last used
substitute pattern (this is vi compatible).  This makes it easy to globally
replace a string:
    :g/pat/s//PAT/g
This replaces all occurrences of "pat" with "PAT".  The same can be done with:
    :%s/pat/PAT/g
Which is two characters shorter!

Why \1 works here, and not anywhere else

As I just mentioned, :s//replacement will re-use the last search pattern. See :help last-pattern for more.

Fortunately, the :g command also sets this "last pattern".

So, we can translate:

:g/aaa\(.*\)/s//X\1X <=> :g/aaa\(.*\)/s/aaa\(.*\)/X\1X

The capture group exists during the substitute. But,

:g/aaa\(.*\)/echo \1 <=> :g/aaa\(.*\)/echo \1

There's no capturing done in the second part of the global command. Vim just runs the literal :echo \1 on every line matching the pattern.

Answer 2

As said by the other answers you can not reuse the search pattern and the subgroups with every command. So here is another general way that works with every command: use :execute, getline() and substitute():

:global /aaa/ execute 'python3 print("' . substitute(getline('.'), 'aaa(.*)', '\1', '') . '"*3)'

Answer 3

It ain't pretty, but it is possible. The following achieves your pseudo code of :g/aaa\(.*\)/python3 print("\1"*3)

:%s/aaa\(.*\)/\=submatch(0) . execute('python print("'.submatch(1).'"*3)')

N.B. Note that, for some reason, when run in this command, while the python print commands are run, their output doesn't seem to make it onto the screen. If you need the output to be displayed, then you can extract the print into a function, which might be nicer for complicated Python code, anyway:

function! RunPython(match)
  python print vim.eval("a:match")*3
  return ""
endfunction

:%s/aaa\(.*\)/\=submatch(0) . RunPython(submatch(1))

(Note that in the function, I'm using Python 2, because that's what I have installed.)

How it Works

We can actually run arbitrary Vimscript from within a :substitute command, as explained by :help sub-replace-special. If you start your replacement string with \=, then it is evaluated as an expression.

As such, all (sic) you need to do is construct an expression that executes some Python code but then evaluates to the value of the entire match (because we don't actually want to perform a replacement). The code above does this by the "elegant" means of concatenating submatch(0) (the whole match) with an execute command that actually runs the Python code but doesn't return any value.

Answer 4

After some time, where I am more experienced, I decided to write a generic solution for this problem. It is based on the answer of Rich

Usage:

:<range>GL/pat/py python code 

Passes over a range that is filtered by [pat]. Executes python code, where the first matched group is stored in a variable called match. Doesn't change anything...

OR:

:<range>GL/pat/rpy python expression 

Passes over a range that is filtered by [pat]. Evaluates a python expression, where the first matched group is stored in a variable called match. Changes the entire match expression to the returned value from the expression.

Example:

:%GL/lst=\(.*\)/rpy 'lst = ' + match 

changes

lst=['aaa']

into

lst = ['aaa']

To use it, you may add the following lines to your vimrc:

function! RunPython2(match,run)
  py3 import vim
  py3 match=vim.eval("a:match")
  py3 retval=eval(vim.eval("a:run"),globals())
  py3 if retval==None: retval=match
  py3 vim.command("let retInVim='" + retval + "'") 
  return retInVim
endfunction

function! RunPython(match,run)
  py3 import vim
  py3 match=vim.eval("a:match")
  exec "py3 " . a:run 
  return ""
endfunction

function! GL(arg) range
    let lst=matchlist(a:arg,'/\(.*\)/\(.\{-}\) \(.*\)$')
    if lst[2]==#"rpy"
        exec a:firstline. "," . a:lastline . ":s\/" . lst[1] . "/\\=RunPython2(submatch(1),\"". escape(lst[3],"\"") . "\")"
    elseif lst[2]==#"py"
        exec a:firstline. "," . a:lastline . ":s\/" . lst[1] . "/\\=submatch(0) . RunPython(submatch(1),\"". escape(lst[3],"\"") . "\")"
    endif
endfunction 
command! -nargs=1 -range GL <line1>,<line2>call GL(<f-args>)