5

From :h /\+:

\+  Matches 1 or more of the preceding atom, as many as possible.

And :h /\{:

\{n,}   Matches at least n of the preceding atom, as many as possible

It seems the multi + and {1,} have the same meaning.

But if I have a buffer containing 4 identical lines and a fifth different line:

some random text
some random text
some random text
some random text
bar

And I type the following substitution command:

:%s/\v(.*\n)\1+//

Or this one:

:%s/\v(.*\n)\1{1,}//

I don't get the same result.

They are almost identical, the only difference is that one uses the multi + while the other uses {1,}. So I should get the same result.

The second one works as expected, because it removes the four lines some random text.
But the first one doesn't work as expected, because it keeps one of the four some random text.

It seems the problem comes from the regex engine, because by default Vim uses the new NFA engine, but if I ask Vim to use the old one by prefixing the pattern with \%#=1, I get the expected result:

:%s/\%#=1\v(.*\n)\1+//    removes all 'some random text' lines when using old engine
:%s/\%#=2\v(.*\n)\1+//    keeps a 'some random text' line when using new engine

Is it a bug or does it simply mean that the new engine doesn't support repeating a backref with the multi +?

In the latter case, shouldn't Vim automatically select the old engine?

If it doesn't, where can I find information on what the new regex engine does not support?


In case it matters, the value of 'regexpengine' is 0, which means, according to the help, that Vim can automatically switch the engine if the default one becomes too costly.


@romainl: I was trying to find a substitution command to reduce all consecutive duplicate lines with a single occurrence:

foo          foo
foo    =>    bar
foo          qux
foo
bar
bar
bar
bar
qux

You can do it with this substitution command:

%s/\v^(.*)(\n\1)+/\1/

But in the process of finding this command, I found the behavior described earlier which I don't understand.

If you move the newline \n from the second captured group to the first one (which I know you shouldn't because then it won't manage the case where a duplicate line is at the very end of the buffer), then, depending on whether you use the multi + or {1,}, you don't get the same result.

You can let Vim choose the engine (provided 're''s value is 0), force it to use the old one with \%#=1, or force it to use the new one with %#=2, which makes 3 choices.
You can use the multi + or the multi {1,}, which makes 2 other choices, so 6 possible commands in total:

(1)    %s:\v^(.*\n)\1+:\1:         (4)    %s:\v^(.*\n)\1{1,}:\1:

(2)    %s:\%#=1\v^(.*\n)\1+:\1:    (5)    %s:\%#=1\v^(.*\n)\1{1,}:\1:
(3)    %s:\%#=2\v^(.*\n)\1+:\1:    (6)    %s:\%#=2\v^(.*\n)\1{1,}:\1:

The commands (2), (4) and (5) give the expected result against the previous buffer:

foo
bar
qux

The commands (1), (3) and (6) don't work:

foo
foo
bar
bar
qux

It seems that using the new engine is the problem ((3) and (6) use it explicitly while (1) probably uses it by default).

(2) and (5) work because they use the old engine explicitly. (4) works, probably because it uses the old engine by default (contrary to (1)).

What's weird is that as soon as the newline is moved outside the first captured group, everything works no matter the multi and no matter the engine:

%s:\v^(.*)(\n\1)+:\1:         %s:\v^(.*)(\n\1){1,}:\1:

%s:\%#=1\v^(.*)(\n\1)+:\1:    %s:\%#=1\v^(.*)(\n\1){1,}:\1:
%s:\%#=2\v^(.*)(\n\1)+:\1:    %s:\%#=2\v^(.*)(\n\1){1,}:\1:

All these 6 commands work and perform the exact same transformation.

  • 2
    it simply mean that the new engine doesn't support repeating a backref with the multi +? - Bingo, the new engine doesn't support backtracking. – Sato Katsura Jun 19 '16 at 17:36
  • @SatoKatsura Again, you're right, thank you. I tested with the 2 engines and the 2 multi, and now the behavior is more consistent. It fails with both multi and the new engine, and it works with both multi and the old engine. The only thing I'm not sure to understand, is why by default Vim seems to use the new engine with \1+ but the old with \1{1,}. That and the whole backtracking notion. I thought the new engine could backtrack. For example, if I ask to find .*foo, since * is greedy, .* should consume all the line. And then, I thought the engine had to "backtrack" to match foo. – user9433424 Jun 19 '16 at 18:46
  • @SatoKatsura I must have misunderstood the notion. Anyway, thank you for your answer. – user9433424 Jun 19 '16 at 18:46
2

What exactly are you trying to do with that \1? :%s/\v(foo\n)+// works exactly as expected.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.