Grabbing values between quotation marks

Question

I want to create a regex that matches strings within single or double quotes in vim and that considers the fact that some strings might have escaped single quotes or double quotes.

So far, I've found this answer in an stackoverflow question which does the same thing but in Javascript regex flavor. By using the logic of that regex, I tried to write a regular expression but in vim regex flavor. The reasoning I followed for translating the javascript regex to a vim regex is explained below

(["'])(?:(?=(\\?))\2.)*?\1

• (["'])

  • Definition: capture group of single or double quotes
  • vim equivalent: (["'])

• (?:(?=(\\?))\2.)

  • Definition: Non-capturing group of (?=(\\?))\2.
  • vim equivalent: %((\\?)@=\2.)
  • (?=(\\?))
    • Definition: Positive lookahead of (\\?)
    • vim equivalent: (\\?)@=
  • \2
    • Definition: Backreference to \\?
    • vim equivalent: \2
  • .
    • Definition: Matches any character different from line breaks
    • vim equivalent: .
• *?
  • Definition: Non-greedy operator of (?:(?=(\\?))\2.)
  • vim equivalent: {-}
• \1
  • Definition: Backreference to ["']
  • vim equivalent: \1

By following the reasoning explained above, the following Javascript regex

(["'])(?:(?=(\\?))\2.)*?\1

would be equivalent to the following vim regex

/\v(["'])%((\\?)@=\2.){-}\1

However, this regex does not completely mimick the behavior of the Javascript regex, because it matches some patterns (see image below) that are not matched by the Javascript regex presented above (see what the latter matches here).

Why is the vim regex matching any character that follows a single quote or a double quotes given that the last character of the regex is a backreference to a single quote or a double quote?

Answer 1

You can solve that with this regex:

\v(['"])%(\1@![^\\]|\\.)*\1

Which will solve all of your test cases, plus others such as "\\" or '\\'.

Explaining it:

• \v: Use "verymagic" mode, so we don't need to backslash parens, etc.
• (['"]): Matches either a single or double quote and store it in \1.
• %(...)*: Then match zero or more of:
• \1@![^\\]: A single character that does not match the quote itself (\1@! is a negative lookahead for whatever is in the capture group) or a backslash. So any character other than those. (Or a newline, that's also not matched by the [...] expression.)
• |: Or...
• \\.: A backslash followed by any character. Including a second backslash, or the quote, whatever the quoting style was.
• \1: Finally, a matching closing quote.

The trick is to "skip" the character following the escaping backslash, whatever it is. This also has the effect that pairs of backslashes are fine, before the closing quote.

The negative lookahead in this regex might turn out to be somewhat inconvenient and could make it expensive to run. An alternative is to expand it into two separate matches, for each style of quotes:

 \v'%([^'\\]|\\.)*'|"%([^"\\]|\\.)*"

That's exactly the same idea, just using the fact that we can shorten the single character match to [^'\\] or [^"\\], since we know exactly which quote was used to open the string in this case. We also no longer need the capturing group and the backreference.

Tested to match all of:

"a"
'a a'
'a"a'
"a''a"
"a'\"\"'a"
"\"\"\""
"\\"
""
''

And not to match any of:

"a
"\"
"a'
'a"
'\\\'

Answer 2

In the department of simplified versions this may do...

\v(['"]).{-}[^\\]\1

I say "may" as I wouldn't stake my life on there being zero failing corner cases but it passes your tests and and every other I've tried so far.

Using "very magic" that's...

• (['"]) - capture a quote
• .{-} - non-greedy slurp (of all but two characters...see next)
• [^\\]\1 - match non-escaped quote of the same type as the opener

Seems almost too simple! But avoiding the potential performance hits of things like look(ahead|behind)s, if possible, is a good thing.

You may have noticed that this ignores "empty strings" ('', ""). If we want to include them there's still no need to resort to complex atoms. Alternations, often overlooked as a KISS option, should work here. Something like...

(''|""|(['"]).{-}[^\\]\2)

Update: A comment mentioned a corner case issue: pairs of backslashes. If you're concerned about that case this'll solve it. (Still look(ahead|behind) free but getting a little longer):

(['"])(\\(\\|\1)|[^\\]){-}[^\\]*\1

With empty string matching:

(""|''|(['"])(\\(\\|\2)|[^\\]){-}[^\\]*\2)