Run global substitute for each match of a regex, with increment

Question

I've got a somewhat obfuscated javascript file with multiple variables named a##########, where # stands for a single 0-9 digit. Each such identifier may or may not be unique.

I was substituting these manually, using :%s/<c-r>k/f#/g, where k was the register I put one of these identifiers and # was a number I was incrementing manually.

This soon proved quite slow and tedious, as well as error-prone, so I decided to do them all at once, automatically. I tried the help files and online and came up with this (after I'd named the first 30 occurrences manually):

let: counter = 31 | g/a\d\{10\}/s//\="f".counter/ | let counter = counter + 1

This worked, except instead of running each substitution globally, it ran it for each match individually, thus distorting the meaning of the identifiers (e.g. where there would be a234789453 5 times, it would give me f1, f2 etc. for each match, instead of replacing all instances of the same match with the same text).

So I ran the same, placing the regex in s's call and using the g flag for s and got the same results, whether I made these modifications to the original command alone or separately. Then I tried:

let: counter = 31 | %s/a\d\{10\}/\="f".counter/ | let counter = counter + 1

Which resulted in all occurences of the pattern being replaced by f31, regardless of the g flag or not.

Clearly, the issue is that I need to run %s/__match__/\="f".counter/g for each match specifically, but I can't seem to find a way to explain this to vim.

Is there a way to run a global substitute for each unique match of a regex?

Answer 1

While the other solutions involving VimL should work just fine, the simple (and to some extent straightforward) approach is to use a macro. The trick is to make the macro recursive, to replace all variables automatically:

• :let cnt = 31Enter - initialize cnt
• qqq - clear register q
• qq - start recording a macro named q
• /\v<a\d{10}>Enter - search for variable names
• * - set the search register @/ to match the current variable
• :%s//\='f'.cnt/gEnter - humanize the current variable
• :let cnt += 1Enter - update cnt for the next variable
• @q - recursively invoke the current macro; the macro stops at the first error, namely when /\v<a\d{10}> no longer matches anything
• q - finish recording the macro.

Then you'd just run the macro: @q.

Answer 2

Although I think John's answer is quite good, I thought of a different way to achieve this. It still requires a custom function, but it is a little bit simpler.

The idea is to create an incrementer object that keeps track of the individual variable IDs. To do this, we define a dictionary with an incrementer function (you can add this for example to your vimrc):

let g:incrementer = {
      \ 'n' : 31,
      \ 'ids' : {},
      \ }

function! g:incrementer.increment(id)
  if !has_key(self.ids, a:id)
    let self.ids[a:id] = self.n
    let self.n += 1
  endif

  return 'f' . self.ids[a:id]
endfunction

With the above code loaded, you can perform the substitution similar to how you tried it yourself:

:%s/a\d\{10\}/\=g:incrementer.increment(submatch(1))/g

Answer 3

I couldn't find a simple one-liner with the functionality you want, but I did cook up a script for it:

function! Replacer(regexp, start)
    let elems = []
    " Loop through all matches of regexp
    call cursor(1, 1)
    let [lnum,col] = searchpos(a:regexp, 'cW')
    while [lnum,col] != [0,0]
        " if match isn't already in the list, add it
        if count(elems, expand("<cword>")) == 0
            call add(elems, expand("<cword>"))
        endif
        let [lnum,col] = searchpos(a:regexp, 'W')
    endwhile
    " Set initial counter
    let counter = a:start
    " For each element, replace with new token
    for elem in elems
        exe '%s/'.elem.'/\="f".counter/g'
        let counter += 1
    endfor

endfunction

You would then do :call Replacer('a\d\{10\}', 31)

How it works

As you saw with your two commands, a global command marks by line instead of by match, and the whole file replacement completes the full replace before doing any increments to your counter.

What we need is to store a list of unique matches, and then iterate through the matches doing a whole file replacement of that match, incrementing the counter as we go.

First, we create a variable to track our matches. We want it to be a list. I called it elems: let elems = []

Next, we want to walk through the file searching for the regexp. First, we need to position the cursor at the top of the file, so we call cursor(1, 1).

Then we start our loop. Each time we'll call searchpos(. We use the W flag to prevent wrap-around, so that our loop ends at the end of the file. On the very first invokation, we also add the c flag to catch if the regexp matches the very beginning of the file.

At each return of searchpos the cursor has moved to the match. I made an assumption that the regexp is going to match the whole or beginning of a word that we want to replace, instead of replacing just the match. So, we check if the <cword> is already in our elems list using the count function. If it isn't, we add it using the add function.

Now that we have our list of unique matches, we set the counter to the desired start value let counter = a:start, where the start was passed as an argument.

Then, we iterate over our list with a basic for loop for elem in elems and build up a whole-file replacement command with exe. I could have built up the replacement text directly, as in /f' . counter . '/g' but I wanted to keep the replacement closer to your example, so I only inserted the value of elem into the search, and let the replacement be the expression "f".counter

Increment the counter after each full replacement counter += 1 and we are done.

Example run

Given the file:

int main() {
    int a1234567890;
    int a4820510384;

    a1234567890 = 5;
    a4820510384 = 7;

    a4820510384 += a1234567890;

    return a1234567890*a4820510384 + a4820510384;
}

(sorry for the example being a c file isntead of javascript; the replacement should work just as well for javascript or any language where cword behaves appropriately)

running

:call Replacer('a\d\{10\}',31)

shows

4 substitutions on 4 lines
5 substitutions on 4 lines

and gives

int main() {
    int f31;
    int f32;

    f31 = 5;
    f32 = 7;

    f32 += f31; 

    return f31*f32 + f32;
}

Answer 4

Many linux distributions offer a Vim with Perl support. If this is the case we can use :perl perl-command and :perldo perl-command:

:perldo $c=31
:perldo s/a(\d{10})/$m{$1} ||= $c++; "f$m{$1}" /ge

(untested)

The solution presented is in fact very similar to the question...

• :perldo perl-command executes perl-command for the range (def: all lines)
• if "perl-command" modifies the default string, the line gets modified. Perl's s/regExp/str/ is very similar to vim's s///...
• but "eval" option: s/RegExp/perl-exp/e is nice and powerful: it substitutes by eval-in-perl(perl-exp), in our case:
  • for all occurences of a\d{10}
  • define a new number if we have a new case: $m{number} = $n{number} || $c++
  • replace by fNUMBER
• RegExp (following the question) is "a" followed by 10 digits (\d{10})