1

I have a syntax whereby #J indicates embedded JSON. This can be followed by any JSON object such as #J[1,2,3]. The atom cases (Boolean, null, number, string) are handled as a json_atom region. This is a bit of a hack which says that #J, followed by optional whitespace, begins a region which ends on a whitespace character or closing parenthesis/bracket/brace: and the closing character is excluded.

syn region json_atom matchgroup=Delimiter start="#J[\t\n ]*"rs=e end="[\t\n \])}]"re=e-1 contains=[... various things ...]

One of the things it contains is this is json_keyword:

syn keyword json_keyword contained true false null

Now here is the issue: everything looks good if when there is whitespace:

#J true

However, when there is no separation

#Jtrue

the keyword is not recognized! A fallback pattern flags the true part in red inverse video as invalid syntax; the #J is colored correctly as a Delimiter.

Even though I used the rs= syntax to set the region start correctly, Vim insists that the J is part of the word, and does not recognize Jtrue.

(This is Vim 8.0, which dates back to 2016; coming from Ubuntu 18).

1

I hacked it like this, by switching the keywords to a match rather than keyword definition:

syn match json_keyword "true\|false\|null" contained

This is okay because the number of keywords is small, and there are no words in JSON that are not keywords. So that is to say, now if we enter, say

#Jfalsehood

the false part is highlighted as a keyword, and hood is flagged inverse red as erroneous. The original idea was that the whole word would be flagged, since falsehood isn't a keyword, but this is not bad; the user can see that a fragment of the word is a keyword.

Incidentally, the unwanted behavior that this question is about can be reproduced with syn match too, like this:

syn match json_keyword "\<\(true\|false\|null\)\>" contained

If we anchor these matches to the start and end of a word, Vim does not recognize them after #J, even though there is a region start boundary there.

It looks like a bug in Vim: Vim is doing the matching against a string which contains the full text, including the previous J, and does not respect the region boundary. The word anchor \< should match at the start of the region, but if the previous character is alphabetic, then it fails. Arguably, the matching of items contained in a region should take place as if no text exists before and after the region.

6
  • 1
    but J and t are both 'iskeyword' characters (I assume), so \< can never match between them in Jt... (:help /\<) – D. Ben Knoble Jun 1 at 22:05
  • @D.BenKnoble You don't seem to acknowledge my point that, notwithstanding iskeyword and \< being well understood and good, J is part of the delimiter for a syntax highlighting region, which starts after the J; and is correctly highlighted on its own. Items in the region should probably be matched as if they are not preceded by any text; the J must be deemed not to exist when we consider whether true is a keyword. There must be no Jt clump exposed to the regex engine or any other matching logic. – Kaz Jun 1 at 22:10
  • 2
    I think the simplest explanation is that word-boundary =/= region boundary. Attempting to put them together seems more confusing, to me. – D. Ben Knoble Jun 1 at 22:14
  • @D.BenKnoble You don't seem to acknowledge that there is a defined region which contains items. One of those items is the keyword true. Vim is saying that the region does not contain this keyword, which doesn't seem reasonable; the region starts with those characters, which end on a word boundary. In almost no syntax are you going to have a region which starts in the middle of a token; it doesn't make sense. – Kaz Jun 1 at 22:40
  • 2
    Perhaps we’re talking past each other, so I’ll disengage after this, but a keyword has word boundaries, and that isn’t met when you use keyword or \<. This is practically orthogonal to syntax regions, AFAIK. – D. Ben Knoble Jun 1 at 22:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.