You have a few problems with your regex that I can see:
- the zero-width forward lookup with
@= covers essentially all of the match, so highlighting your matches only gets the commas or beginning of lines.
\| is splitting the two alternatives in a way that the first (
[^''][^,]*) isn't really anchored to the end (
\%(,\|$\)), so it doesn't match all the way to a comma or end of line.
[^''] match is too weak, in fact it can match a whitespace or a comma, it should be more restrictive.
I tweaked your regex and got to this:
let pattern = '\m\%(^\|,\)\s*\zs\([^'' \t,][^,]*\|''\%(''''\|[^'']\)*''\)\ze\s*\%(,\|$\)\@='
\ze to match the strings themselves, which makes the pattern useful for use in a search. It's also friendly to
You can change that if you're planning to use it differently (for example, in
split() or similar.)
Same as you, I'm starting at the beginning of the line, or a comma. Then skip whitespace and then branch into either a literal or a single quoted string. The match for the first character of the literal is stronger, we reject
' but also whitespace (space or tab) and a comma. (From the YAML specification, we could decide to reject even more characters here.)
After matching one of those, we skip whitespace again (if needed) and then doing a zero-width forward lookup for a comma or the end of the string. (The forward lookup can be important if you're searching repeatedly with
n, since you want the comma to match again on the next use of the pattern.)
One last suggestion I'd have is to use "very magic" mode for the regular expression, which lets us get rid of 13 backslashes in total!
let pattern = '\v%(^|,)\s*\zs([^'' \t,][^,]*|''%(''''|[^''])*'')\ze\s*%(,|$)@='
I still see two issues with this pattern:
- If a literal has trailing whitespace, it will be returned as part of the expression. Perhaps we could tighten the first regex even more to ensure the last matched character will always be a non-blank character.
- When the string doesn't follow the correct syntax, the match may be misleading. For example, if there are literal characters after a quoted string that has a comma inside it, it will match the comma and think that the second part (including a single quote on its own) will form a literal. Depending on how you plan to use this regex (for example, calling
matchstrpos() repeatedly?), you can use stronger anchors that will help you catch invalid syntax.
I tested the pattern with a search, by enabling highlighting, setting the search register and pressing
:let @/ = pattern
I recommend this as a way to debug regular expressions such as this one.
UPDATE: Since the problem is splitting a YAML list rather than using the pattern in a search, we need to take a slightly different approach.
split() is problematic, since the regex matches are not anchored, so it's really hard to match a comma that's guaranteed to be outside of a quoted string.
A better approach is to call
matchlist() repeatedly, always starting the match at the position we last stopped.
In that case, we can always anchor the pattern on
matchlist() with a position will match that to that specific position, not the very beginning of the string.) We should also consume the ending comma and whitespace as we match. Since we now need the full match (to advance the position counter), we'll use a capture group to get the part that interests us, which is the YAML literal or quoted string.
Here's an implementation of this method:
let pattern = '\v^\s*([^'' \t,]%([^,]*[^ \t,])?|''%(''''|[^''])*'')\s*%(,|$)'
let result = 
let pos = 0
let end = len(a:expr)
while pos < end
let match = matchlist(a:expr, pattern, pos)
throw "Parse error, can't match at position ".pos
let [mstr, mgroup; _] = match
call add(result, mgroup)
let pos += len(mstr)
This function will also be able to detect malformed items (such as a quoted string with trailing characters) and it will
throw an error in that case, indicating the position in which the syntax error was found.